Figure 1.
Examples of point clouds from the PCN dataset colored by height. The incomplete point cloud inputs are shown on the left, and the corresponding completed point clouds are shown on the right. (a) N = 2048, (b) N = 16384, (c) N = 2048, (d) N = 16384, (e) N = 2048, (f) N = 16384.
Figure 1.
Examples of point clouds from the PCN dataset colored by height. The incomplete point cloud inputs are shown on the left, and the corresponding completed point clouds are shown on the right. (a) N = 2048, (b) N = 16384, (c) N = 2048, (d) N = 16384, (e) N = 2048, (f) N = 16384.
Figure 2.
A single scene from DALES Viewpoints Version 2, colored by height, with occlusions from multiple flight patterns. (a) Northwest to Southwest, (b) Southwest to Southeast, (c) Southeast to Northwest, (d) Northeast to Northwest.
Figure 2.
A single scene from DALES Viewpoints Version 2, colored by height, with occlusions from multiple flight patterns. (a) Northwest to Southwest, (b) Southwest to Southeast, (c) Southeast to Northwest, (d) Northeast to Northwest.
Figure 3.
Examples of corresponding scenes from the DALES Viewpoints 2 data set, colored by height. The occluded input is in the left-hand column, the desired output is in the middle, and the combined full point cloud is on the right.
Figure 3.
Examples of corresponding scenes from the DALES Viewpoints 2 data set, colored by height. The occluded input is in the left-hand column, the desired output is in the middle, and the combined full point cloud is on the right.
Figure 4.
Visual example local eigenvalues representing different spatial properties on the DALES Viewpoints Version 2 dataset. The original input point cloud is colored by height. The Eigen feature representations are colored by value, with warm tones indicating high values and cool tones indicating low values.
Figure 4.
Visual example local eigenvalues representing different spatial properties on the DALES Viewpoints Version 2 dataset. The original input point cloud is colored by height. The Eigen feature representations are colored by value, with warm tones indicating high values and cool tones indicating low values.
Figure 5.
The diagram of the proposed hierarchical sampling algorithm is shown above. The input is our set of points P and the output is the downsampled results Q.
Figure 5.
The diagram of the proposed hierarchical sampling algorithm is shown above. The input is our set of points P and the output is the downsampled results Q.
Figure 6.
This figure shows a diagram of the variance-based feature selection. After the Eigen features are calculated, using a combination of local eigenvalues and MLPs, we obtain a feature descriptor for each point in the point cloud. We remove any point with low variance in the descriptor from consideration and then perform a max-pooling operation. The selected features are those with the highest feature response.
Figure 6.
This figure shows a diagram of the variance-based feature selection. After the Eigen features are calculated, using a combination of local eigenvalues and MLPs, we obtain a feature descriptor for each point in the point cloud. We remove any point with low variance in the descriptor from consideration and then perform a max-pooling operation. The selected features are those with the highest feature response.
Figure 7.
Visual comparison of farthest point sampling and our proposed Eigen feature sampling. Each point cloud is colored by height. The original input point cloud is shown in the right-hand column at the actual resolution of 2048 points. The center column shows downsampling to 1024 points using farthest point sampling. The right-hand column shows our Eigen feature sampling, also at 1024. (a) N = 2048, (b) N = 1024, (c) N = 1024, (d) N = 2048, (e) N = 1024, (f) N = 1024.
Figure 7.
Visual comparison of farthest point sampling and our proposed Eigen feature sampling. Each point cloud is colored by height. The original input point cloud is shown in the right-hand column at the actual resolution of 2048 points. The center column shows downsampling to 1024 points using farthest point sampling. The right-hand column shows our Eigen feature sampling, also at 1024. (a) N = 2048, (b) N = 1024, (c) N = 1024, (d) N = 2048, (e) N = 1024, (f) N = 1024.
Figure 8.
Example of DALES Viewpoints scenes processed with a Gaussian mixture model. The left-hand side is the original scene colored by height. The middle column shows the scene clusters, colored with a random RGB combination for each unique cluster. Finally, the right-hand side shows the resampled scenes, colored by height. These scenes have 100 components each.
Figure 8.
Example of DALES Viewpoints scenes processed with a Gaussian mixture model. The left-hand side is the original scene colored by height. The middle column shows the scene clusters, colored with a random RGB combination for each unique cluster. Finally, the right-hand side shows the resampled scenes, colored by height. These scenes have 100 components each.
Figure 9.
The proposed architecture is shown above. Loss is calculated between each feature layer, and we compare the feature layers using an MSE loss. This feature loss is added to the overall loss.
Figure 9.
The proposed architecture is shown above. Loss is calculated between each feature layer, and we compare the feature layers using an MSE loss. This feature loss is added to the overall loss.
Figure 10.
Example scenes of the reconstructed point clouds from our method on the DALES Viewpoints Version 2 dataset. All point clouds are colored by height.The images show the input point cloud, with occlusions on the left-hand side, with the right hand-side showing the same point cloud with additional points added by our network to fill the occluded area.
Figure 10.
Example scenes of the reconstructed point clouds from our method on the DALES Viewpoints Version 2 dataset. All point clouds are colored by height.The images show the input point cloud, with occlusions on the left-hand side, with the right hand-side showing the same point cloud with additional points added by our network to fill the occluded area.
Figure 11.
Progression of the reconstruction operation on the DALES Viewpoints Version 2 dataset. All point clouds are colored by height. The network begins to identify the occlusion areas in the initial epochs and then refines the details in later epochs: (a) 10 epochs, (b) 50 epochs, (c) 100 epochs, (d) 150 epochs, (e) 200 epochs, (f) 250 epochs, (g) 300 epochs, (h) Ground Truth.
Figure 11.
Progression of the reconstruction operation on the DALES Viewpoints Version 2 dataset. All point clouds are colored by height. The network begins to identify the occlusion areas in the initial epochs and then refines the details in later epochs: (a) 10 epochs, (b) 50 epochs, (c) 100 epochs, (d) 150 epochs, (e) 200 epochs, (f) 250 epochs, (g) 300 epochs, (h) Ground Truth.
Figure 12.
The above image shows examples of areas of occlusion that have been reconstructed with our method using the DALES Viewpoints Version 2 dataset. All point clouds are colored by height. The initial input point clouds are in the left-hand column, our predicted point clouds are in the middle column, and the right-hand column depicts the ground truth.
Figure 12.
The above image shows examples of areas of occlusion that have been reconstructed with our method using the DALES Viewpoints Version 2 dataset. All point clouds are colored by height. The initial input point clouds are in the left-hand column, our predicted point clouds are in the middle column, and the right-hand column depicts the ground truth.
Figure 13.
The above image shows examples of our method when applied to the PCN dataset. All point clouds are colored in the horizontal direction. Original occluded point clouds are on the left-hand side, the predicted clouds are in the center column, and the ground-truth point clouds are in the right-hand column.
Figure 13.
The above image shows examples of our method when applied to the PCN dataset. All point clouds are colored in the horizontal direction. Original occluded point clouds are on the left-hand side, the predicted clouds are in the center column, and the ground-truth point clouds are in the right-hand column.
Figure 14.
These images show examples of the scenes from Dataset 2. Dataset 2 contains the initial occluded input point clouds and our generated points. These have been combined and then run through a semantic segmentation network. Each scene has 18,882 total points with eight classes. Each point is labeled by object category: ground (blue), vegetation (dark green), power lines (yellow), poles (dark orange), buildings (red), fences (light orange), trucks (light green), cars (light green), unknown (dark blue).
Figure 14.
These images show examples of the scenes from Dataset 2. Dataset 2 contains the initial occluded input point clouds and our generated points. These have been combined and then run through a semantic segmentation network. Each scene has 18,882 total points with eight classes. Each point is labeled by object category: ground (blue), vegetation (dark green), power lines (yellow), poles (dark orange), buildings (red), fences (light orange), trucks (light green), cars (light green), unknown (dark blue).
Figure 15.
These images show examples of the scenes from Dataset 2. Dataset 2 contains the initial occluded input point clouds and our generated points. These have been combined and then run through a semantic segmentation network. Each scene has 18,882 total points with eight classes. Each point is labeled by object category: ground (blue), vegetation (dark green), power lines (yellow), poles (dark orange), buildings (red), fences (light orange), trucks (light green), cars (light green), unknown (dark blue).
Figure 15.
These images show examples of the scenes from Dataset 2. Dataset 2 contains the initial occluded input point clouds and our generated points. These have been combined and then run through a semantic segmentation network. Each scene has 18,882 total points with eight classes. Each point is labeled by object category: ground (blue), vegetation (dark green), power lines (yellow), poles (dark orange), buildings (red), fences (light orange), trucks (light green), cars (light green), unknown (dark blue).
Table 1.
Overall results comparing our method to current state-of-the-art point cloud completion methods on the DALES Viewpoints Version 2 dataset.
Table 1.
Overall results comparing our method to current state-of-the-art point cloud completion methods on the DALES Viewpoints Version 2 dataset.
Overall Results: DALES Viewpoints Version 2 Dataset |
---|
Method | Mean CD ↓ | Mean EMD ↓ |
TopNet [16] | 0.002167 | 0.071537 |
PCN [14] | 0.001802 | 0.068283 |
ATLASNet [51] | 0.000474 | 0.067515 |
PointNetFCAE [16] | 0.000468 | 0.112022 |
SA-Net [52] | 0.000433 | 0.039664 |
FoldingNet [53] | 0.000424 | 0.097007 |
Ours | 0.000375 | 0.035604 |
Table 2.
Point cloud completion comparison on Point Cloud Completion dataset in terms of per point Chamfer distance (lower is better).
Table 2.
Point cloud completion comparison on Point Cloud Completion dataset in terms of per point Chamfer distance (lower is better).
Overall Results: Point Cloud Completion Network Dataset |
---|
Methods | Mean | Plane | Cab. | Car | Chair | Lamp | Couch | Table | Boat |
AtlasNet [51] | 17.69 | 10.37 | 23.4 | 13.41 | 24.16 | 20.24 | 20.82 | 17.52 | 11.62 |
FoldingNet [53] | 16.48 | 11.18 | 20.15 | 13.25 | 21.48 | 18.19 | 19.09 | 17.8 | 10.69 |
PCN [14] | 14.72 | 8.09 | 18.32 | 10.53 | 19.33 | 18.52 | 16.44 | 16.34 | 10.21 |
TopNet [16] | 9.72 | 5.5 | 12.02 | 8.9 | 12.56 | 9.54 | 12.2 | 9.57 | 7.51 |
SA-Net [52] | 7.74 | 2.18 | 9.11 | 5.56 | 8.94 | 9.98 | 7.83 | 9.94 | 7.23 |
Ours | 7.10 | 2.51 | 10.29 | 10.29 | 8.07 | 6.54 | 6.64 | 6.61 | 5.87 |
Table 3.
Comparison of the proposed sampling method against FPS and SampleNet learned sampling. Lower is better.
Table 3.
Comparison of the proposed sampling method against FPS and SampleNet learned sampling. Lower is better.
Ablation Study: Eigen Feature Sampling |
---|
Sampling Method | Mean CD ↓ | Mean EMD ↓ |
FPS | 0.00208 | 0.07594 |
SampleNet [38] | 0.00282 | 0.13035 |
Eigen Feature Sampling (Ours) | 0.00184 | 0.05244 |
Table 4.
Overall results on the DALES Viewpoints dataset comparing the original SA-Net implementation with the same implementation and our proposed point correspondence loss.
Table 4.
Overall results on the DALES Viewpoints dataset comparing the original SA-Net implementation with the same implementation and our proposed point correspondence loss.
Ablation Study: Point Projection for Stabilizing Point Correspondences |
---|
Method | Mean CD ↓ | Mean EMD ↓ |
Single Network | 0.00208 | 0.07594 |
Parallel Network (Ours) | 0.00167 | 0.04951 |
Table 5.
Comparison of the overall accuracy and mean IoU of the semantic segmentation results using the PointNet++ architecture. Dataset 1 refers to the DALES Viewpoints Version 2 dataset using only the input points containing occlusions. Dataset 2 refers to the DALES Viewpoints Version 2, which includes the input points with occlusions, supplemented by our generated points.
Table 5.
Comparison of the overall accuracy and mean IoU of the semantic segmentation results using the PointNet++ architecture. Dataset 1 refers to the DALES Viewpoints Version 2 dataset using only the input points containing occlusions. Dataset 2 refers to the DALES Viewpoints Version 2, which includes the input points with occlusions, supplemented by our generated points.
Semantic Segmentation: Overall Results |
---|
| Overall Accuracy | Mean IoU |
Dataset 1 | 0.685 | 0.395 |
Dataset 2 | 0.865 | 0.451 |
Table 6.
Comparison of per class IoU, semantic segmentation results using the PointNet++ architecture. Dataset 1 refers to the DALES Viewpoints Version 2 dataset using only the input points containing occlusions. Dataset 2 refers to the DALES Viewpoint Version 2, which includes the input points with occlusions, supplemented by our generated points.
Table 6.
Comparison of per class IoU, semantic segmentation results using the PointNet++ architecture. Dataset 1 refers to the DALES Viewpoints Version 2 dataset using only the input points containing occlusions. Dataset 2 refers to the DALES Viewpoint Version 2, which includes the input points with occlusions, supplemented by our generated points.
Semantic Segmentation: Per Class IoU |
---|
| ground | buildings | cars | trucks | poles | power lines | fences | veg |
Dataset 1 | 0.740 | 0.713 | 0.266 | 0.262 | 0.204 | 0.660 | 0.148 | 0.556 |
Dataset 2 | 0.871 | 0.724 | 0.245 | 0.256 | 0.214 | 0.667 | 0.152 | 0.769 |