Effective Planar Cluster Detection in Point Clouds Using Histogram-Driven Kd-Like Partition and Shifted Mahalanobis Distance Based Regression
Round 1
Reviewer 1 Report
In this manuscript, the author introduced an effective planar clustering strategy in point clouds while using histogram-driven kd-like partition and shifted Mahalanobis distance-based regression. The experimental results indicate the superior performance of the proposed approach when compared to the existing benchmark algorithms. The paper is prepared in relatively good shape. I only have some minor comments for it.
Line 311: My understanding is the time complexity for calculating histogram is O(n). Then, the time complexity to find the peak in the histogram is O(n). In addition, since the depth of the tree is log(n), the total time complexity for histogram generation and kd-like partition is O(n+nlogn) -> O(nlogn). Can you please clarify this? Section 4: Methodology or Results? Equations 13 and 14: How did you decide if an extracted planar cluster is correctly segmented or not? Did you set a threshold for correctly segmented planar clusters (e.g., 90% correctly segmented points compared to the ground-truth)? Have you got any chance to test the proposed approach on some outdoor-scene point clouds?Author Response
Responses to Reviewers’ comments for:
“Effective planar cluster detection in point clouds using histogram-driven kd-like partition and Shifted Mahalanobis Distance based regression”
Manuscript ID: remotesensing-571706
Dear Editors and Reviewers,
thank you for invaluable comments regarding our paper (remotesensing-571706 ). We do appreciate the possibility of our manuscript re-submission after extensive revision. Thus, the manuscript was carefully revised following the reviewers’ comments. Additionally, we provide responses to the comments point-by-point.
Response to Reviewer 1 comments
Line 311: My understanding is the time complexity for calculating histogram is O(n). Then, the time complexity to find the peak in the histogram is O(n). In addition, since the depth of the tree is log(n), the total time complexity for histogram generation and kd-like partition is O(n+nlogn) -> O(nlogn). Can you please clarify this?
Thank You for a possibility to clarify this issue. You are right. The exact number of computations is n+nlogn, however, Big O() notation assumes to present time complexity up to its “worst” part. If a number of computations is n+nlogn, the complexity is referred to as “superlinear” O(nlogn). While n increases, log-linear part (nlogn) becomes dominant and the linear part (n) can be neglected.
Section 4: Methodology or Results? Equations 13 and 14: How did you decide if an extracted planar cluster is correctly segmented or not? Did you set a threshold for correctly segmented planar clusters (e.g., 90% correctly segmented points compared to the ground-truth)? Have you got any chance to test the proposed approach on some outdoor-scene point clouds?
In our research we focused on indoor scenes since human-made objects and architecture may indeed be easily decomposed to geometric primitives (planes in its simplest form). Outdoor scenes were not of our interests in this research. Outdoor scenes usually contain natural objects which cannot be easily disassembled into planes due to low point cloud density that outdoor scenes suffer from. We have clarified this issue in the manuscript.
Concerning decision about correct-not correct clusters. The assignment is done taking the maximum overlapping part with respect to the output cluster, not less than 80% (line 418/419).
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The work deals with plane detection in point cloud data with use of developed method by authors. Planar segmentation is researched by many years. The use of histograms in segmentation, and use of MD in regression is not new. But they introduced new information with combining them by k-tree in existing segmentation methodology.
I would recommend to discuss the following Works as well:
Yusheng Xu, WeiYao, Ludwig Hoegner & UweStilla (2017) Segmentation of building roofs from airborne LiDAR point clouds using robust voxel-based region growing, Remote Sensing Letters, 8:11, 1062-1071, DOI: 10.1080/2150704X.2017.1349961 Anandakumar M. Ramiya, RamaRao
Nidamanuri, Ramakrishan Krishnan, Segmentation based building detection approach from LiDAR point cloud, The Egyptian Journal of Remote Sensing and Space Science, Volume 20, Issue 1, 2017.
**I would suggest to shorten related works part. The paper contains details from the literature. E.g. related works could contain only small details rather than giving all the previous methodology.
** it is difficult to follow paper since there is no methodology flowchart to let reader to understand the method in general, first,
It is good to have algorithm charts for intermediate steps, but I mean a general flowchart is missing.
**The quality analysis is performed with considering the nr of detected planes. I would suggest to repeat it by comparing the boundaries of enclosed segments with calculating omission and commission errors. So this will show not only overlapping, but also the accuracy of the outlines from the proposed method.
**The results are shown from one single room datasets. But in reality, we have several or separate rooms to be segmented. Since RANSAC method has problems with the planes which are separate but on same surface. I would like to see the performance of the proposed method in such cases.
Author Response
Responses to Reviewers’ comments for:
“Effective planar cluster detection in point clouds using histogram-driven kd-like partition and Shifted Mahalanobis Distance based regression”
Manuscript ID: remotesensing-571706
Dear Editors and Reviewers,
thank you for invaluable comments regarding our paper (remotesensing-571706 ). We do appreciate the possibility of our manuscript re-submission after extensive revision. Thus, the manuscript was carefully revised following the reviewers’ comments. Additionally, we provide responses to the comments point-by-point.
Response to Reviewer 2 comments
1. I would suggest to shorten related works part. The paper contains details from the literature. E.g. related works could contain only small details rather than giving all the previous methodology.
Thank you for your suggestion. We shortened the section dedicated to related works in negligible details. Additionally we discussed the works you mentioned in the review. Covariance-based plane model fitting remained extensively explained so as to emphasise our contribution and research motivation in relation to the state-of-the-art approaches.
2. it is difficult to follow paper since there is no methodology flowchart to let a reader to understand the method in general
Following your suggestion we added the flowchart. Current Figure 2 of the manuscript demonstrates method flowchart. Thank You for this comment.
3. The quality analysis is performed with considering the nr of detected planes. I would suggest to repeat it by comparing the boundaries of enclosed segments with calculating omission and commission errors. So this will show not only overlapping, but also the accuracy of the outlines from the proposed method
Thank You for this remark. We resigned from comparing outlines since for semantic scene analysis precise borders do not play as important role as correct area and plane position. Therefore, the maximum overlapping strategy with threshold of 80% was applied, as other authors did in the state-of-the-art experiments. This value ensures sufficient quality of detected planes.
Outlines of planar fragments are studied and widely exploited for point cloud model-based compression methods, but these aspects are beyond the scope of current research. In our further research we plan to deal with suggested aspects.
4. The results are shown from one single room datasets. But in reality, we have several or separate rooms to be segmented. Since RANSAC method has problems with the planes which are separate but on the same surface. I would like to see the performance of the proposed method in such cases.
Indeed, RANSAC suffers from that problem. That is why we did not use RANSAC and we engaged density-based clustering algorithm to separate planar patches belonging to the same mathematical plane model. In order to evaluate precisely the performance of separation of coplanar fragments, adequate ground-truth information is required. Unfortunately, it is neither provided nor it can be easily determined with any well recognised point cloud database. Annotated points’ clusters are usually noised and their coplanarity might be misleading - perceptually coplanar clusters might correspond, in reality, to different plane models.
Evaluation of coplanar fragments separation could be an interesting experiment that might be conducted on artificially generated data which provides reliable ground-truth information about coplanarity. In our further research we will focus also on this topic.
In the context of coplanar clusters separation studies, we have noted that having applied density-based separation a number of planar fragments increased, on average, by 30%. It was a contribution of coplanar clusters separation stage. It proves that density-based separation stage of the method considerably influences overall accuracy of segmentation process and contribute planar clusters segmentation process.
Moreover, we have considered single-room datasets because all of state-of-the-art authors, we followed, have separated point clouds into individual rooms before segmentation experiments - we, as the other authors, assumed this aspect as an assumption of the research. The method is said to be used for a single scene rather than for collective sets of rooms. In fact, point clouds are usually obtained sequentially, for each room separately rather than continuously for all rooms. In case of several rooms within one point cloud they should be initially separated into individual rooms.
Author Response File: Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Paper is interesting.
My specific comments:
in section 2 - related works - very little about comparison to other scientific research centres' experiences.
It would be good to itroduce some infotmration about application of 3D models, that means - civil enginering. In this case point clouds are good in for example: 3D city modeling, interior modeling. You gave some examples of interior segmentation but there is no mentioned about applications of similar methods in civil engineering - it should be placed right in section 2.
I have some doubts about notation of precision (in abstract - 2.3 % and 0.3%) and in table 3 or conclusions. Why did you use this precision of percantage value notation?I think integer is enough in this case. Please explain.
I am not expert in english language, but in my opinion paper requires extended english editting, only some examples:
page 6 line 211: elected - should be chosen
page 14 line 392: apprised holistically - ?
It is necessary to broaden literature review. Also in Polish research centres (eg. Warsaw, Gdansk). You can find authors whose papers lacking in references.
Please see:
Kedzierski M., Fryskowska A., Wierzbicki D., Dabrowska M, Grochala A. Impact of the method of registering terrestrial laser scanning data on the quality of documenting cultural heritage structures, International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences - ISPRS ArchivesVolume 40, Issue 5W7, 2015, Pages 245-248, 25th ISPRS International CIPA Symposium 2015
Markiewicz J.S., The use of computer vision algorithms for automatic orientation of terrestrial laser scanning data, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, Volume XLI-B3, 2016 XXIII ISPRS Congress, 12–19 July 2016, Prague, Czech Republic, DOI: 10.5194/isprsarchives-XLI-B3-315-2016
or
Ziolkowski P., Szulwic J., Miskiewicz M., Deformation Analysis of a Composite Bridge during Proof Loading Using Point Cloud Processing. Sensors, 2018, 18, 4332, DOI: 10.3390/s18124332
.... or others. I am sure that these research centres also describe similar problems.
Reviewer 2 Report
Improved planar cluster detection in an unorganized point cloud using histogram-driven kd-like partition and Multivariate Normal Distribution regression
This paper deals with a planar segmentation approach for Point Clouds based on a bottom-up methodology that takes into account point cloud subvidivision based on a kd-tree
The manuscript is well written and structured. The extensive related work revision makes this paper very academic and informative, but there are well-known and established results in the literature (ie. PCA, centroid, covariance matrix...) that would only need a cite. This passage should be more concise.
The methodology is tested on 3 point clouds:one of them from the case study and 2 random rooms from a database. Conclusions are thought provoking and founded on the results.
Specific remarks:
The goal of the paper is the planar segmentation of the point cloud based on a model fitting approach that makes use of a KdTree-like subdivision. The authors should define in a more precise manner whether they make use only of 3D (x,y,z) geometric data of the points or other features are involved in the study (like color).
Figure 1 shows one of the rooms used for testing, where a large percentage of orthogonal and parallel planar patches are distinguishable.All the test point clouds should be shown and described more in detail if possible.
How would the methodology generalize in absence of orthogonality and presence of random-oriented planar patches?
Line 207-209 Initial otientation procedure should be detailed:
what stands for Axis Aligned Bounding Box (AABB)? A definition or cite would be appropriate.
How are Bx,y and z derived ?
Is this a manual or automatic procedure?
Subsection 3.2.
An image showing the histogram of the point cloud would improve readability of the paper. Please add one.
Line 234 "The peaks of the histograms could indicate the regions where larger planes are contained even though planes are not necessarily orthogonal to coordinate system axes"
Please discuss and revise the assert. The transformation of planar-patches to the histogramdepends on the orientation: an orthogonal plane would transform to a peak in the histogram, but parallel planes would tranform to a constant value in frequency, and all the bins would contain the same number of points.
Does it mean that the presence of orthogonal planes is a constraint for the histogram partition?
Line 248-250. Please detail the neighbourhood of the point to compute the curvature on.
Line 334 -337 Please rewrite this paragraph to improve readability. Please discuss the number of dimensions of the PCA other than 3D. For 3D coordinates, please explain the assert "middle eigenvector are beyond the scope of this study because they do not show general data tendency" To my knowledge, eigenvectors should act as an orthonormal basis, being the vector associated to the shortest eigenvalue the normal vector of the plane, the one associated with the largest eigenvalue the largest dimension of the planar patch and being the third vector orthogonal to the others.
Line 344 Why was five-fold value chosen for normal distribution cover instead of commonly used 2-sigma or 3-sigma (for 95%-98%)?
Line 381: I suggest applying the methodology to the ISPRS indoor modeling benchmark (Khoshelkham et al, 2017) , which contains a number of indoor Point Clouds.
Khoshelham, K., Vilariño, L. D., Peter, M., Kang, Z., & Acharya, D. (2017). THE ISPRS BENCHMARK ON INDOOR MODELLING. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 42.
Reviewer 3 Report
In this paper, the author proposed a plane segmentation approach, which relies on histogram driven kd-like partition and multivariate normal distribution.
Major concerns:
1. The introduction part is very badly presented.
2. The histogram-driven kd-like partition part has to be much clearly presented. Diagrams showing the partition process would be helpful for explaining the proposed method. Meantime, the advantages of using the proposed histogram-driven kd-like partition has to be highlighted. Why is this partition better than other methods? Only because of computational efficiency?
3. The multivariate normal distribution regression is not a new idea. The PCA-based outlier removal using Mahalanobis Distance has been investigated and adopted in various clustering algorithms/applications.
4. Considering the limited technical contribution and quality of presentation, I would suggest to reject this paper.
Other comments:
5. Line 28: “There are four main groups of methods applicable to point cloud segmentation [6]: edge-based, region-growing based, attributes-based and model-fitting based.” However, in the next paragraph, the order is first region-growing-based segmentation, and then edge-based solutions. The orders should be consistent in the two paragraphs.
6. Attributes-based point cloud segmentation is missing in the comparison of different approaches in Introduction.
7. Line 45: What do you mean by currently used methods? RANSAC?
8. Lines 52-53: Please carefully revise the sentence: “Then, selecting the peaks…”.
9. All model fitting-based plane detection approaches introduced in Section 2.1 assume a single model in the involved point cloud? If you are dealing with multiple plane models, what are you going to do?
10. Equation 3: outl means outlyingness? The notation has to be explained.
11. The PCA-based model fitting is not clearly explained. Something like: “for the classical PCA approach, the eigenvector corresponding to the smallest eigenvalue of the variance-covariance matrix defines the plane normal” has to be explained.
12. Line 94: RANSAC is performed through a sampling-and-testing procedure, which has to be repeated until a required number of trials/draws is achieved. The number of trials for RANSAC depends on the percentage of inliers in the dataset. A low percentage of inliers leads to a large number of trials, which can be inefficient. Do you still think RANSAC is the most efficient method for robust model fitting?
13. Line 98: outlying -> outlier
14. Algorithm 1 Line 15: What do you mean by “no bin in H”?
15. Can you please provide some diagrams to explain how the subdivision procedure is conducted?
16. Equation 13: Are you using maximum likelihood estimation? Since you are assuming Gaussian distribution of the points, the maximum likelihood estimation is equivalent to a classic least-squares problem?
17. Table 1: Mahalanobis Distance describes how many standard deviations away the point is from the centroid of the set of points. For removing outliers based on the derived Mahalanobis Distances, an iterative method is usually adopted. For example, in each iteration, if the Mahalanobis Distance from the point to the centroid is greater than a pre-defined threshold (e.g., 3 sigma), the point will be considered as an outlier. Then, the PCA is conducted on all inliers, and the test process is repeated until no outlier can be found. I don’t understand why you choose the four cardinal shifts for outlier removal.
18. Line 370: The threshold of 45 degree is too large?
19. Similar to Hough transform, the adopted aggregation strategy will merge separate plane segments (e.g., gaps between plane segments) with similar normal and intercept into a single cluster, right? Did you separate these segments?
