Reconstruction of Indoor Navigation Elements for Point Cloud of Buildings with Occlusions and Openings by Wall Segment Restoration from Indoor Context Labeling

Abstract

Indoor 3D reconstruction and navigation element extraction with point cloud data has become a research focus in recent years, which has important application in community refinement management, emergency rescue and evacuation, etc. Aiming at the problem that the complete wall surfaces cannot be obtained in the indoor space affected by the occluded objects and the existing methods of navigation element extraction are over-segmented or under-segmented, we propose a method to automatically reconstruct indoor navigation elements from unstructured 3D point cloud of buildings with occlusions and openings. First, the outline and occupancy information provided by the horizontal projection of the point cloud was used to guide the wall segment restoration. Second, we simulate the scanning process of a laser scanner for segmentation. Third, we use projection statistical graphs and given rules to identify missing wall surfaces and “hidden doors”. The method is tested on several building datasets with complex structures. The results show that the method can detect and reconstruct indoor navigation elements without viewpoint information. The means of deviation in the reconstructed models is between 0–5 cm, and the completeness and correction are greater than 80%. However, the proposed method also has some limitations for the extraction of “thick doors” with a large number of occluded, non-planar components.

1. Introduction

With advances in modern surveying and mapping, many research groups have developed various sensor-based measurement techniques [1]. Laser scanning techniques have been very popular [2,3], which can quickly obtain high-precision point cloud information of buildings and extract the indoor building structure [4,5,6]. Although indoor 3D models are of great help, the production process is very complicated. Manual modeling using point cloud is still time consuming [7], and even experienced modelers can obtain significantly different results or make mistakes [8]. Second, since point cloud contain only 3D spatial locations and do not carry semantic information such as spatial usage functions and structural properties [5], they need to be segmented and endowed with semantic information in order to reconstruct indoor navigation elements [5,9,10]. Indoor navigation elements include door, room, corridor etc.

Despite many proposed methods, studies for reconstructing indoor navigation elements are still ongoing [9,10]. With the development of the construction industry, room walls are not sometimes perpendicular to each other. In addition, complex scenes, such as emergency rescue, usually have many occluded objects, which causes missing walls, misclassification of indoor context labels, and incorrect extraction of openings [11,12]. In order to solve these problems, this paper presents a new processing method. In the absence of viewpoint and color information, this method can automatically reconstruct indoor navigation elements from unstructured 3D point cloud of buildings with occlusions and openings. The main contributions of this paper are as follows:

• A method of wall segment restoration using the outline and occupancy information from the horizontal projection of point cloud.
• A method of room segmentation that simulates the work of a laser scanner that not only divides separate rooms, but also distinguishes the connecting transition space between rooms.
• Identify the doors in the room and optimize the connections between the rooms by means of the planimetric projection statistical graph and the given rules.

The remainder of this paper is organized as follows: Section 2 discusses the related research on indoor navigation element extraction. Section 3 presents the proposed method for indoor navigation element extraction and construction. Section 4 validates the proposed method using publicly available datasets. Section 5 evaluates the results. Finally, Section 6 summarizes this study.

2. Related Work

In recent years, the automatic reconstruction of structured 3D models from point cloud is one of the core topics in surveying and mapping. The tasks of building indoor 3D maps through point cloud generally involve indoor boundary extraction, room segmentation and opening detection. Since indoor spaces contain not only the main structure of the building, a large number of factors that affect data collection, such as: furniture, pedestrians, windows, and other data, missing data and a large amount of clutter during data collection can be caused. For this reason, a large number of a priori constraints are usually used to extract information about the building components. For example, the Manhattan world hypothesis, which assumes an environment with horizontal floors and ceilings, and vertical walls that intersect at right angles [13]. However, many architectural elements of the real-world deviate from the Manhattan world assumptions. Much of the work began to introduce the augmented Manhattan hypothesis, the Atlanta world [14], into the work, where there is no restriction of walls intersecting angles compared to the Manhattan world.

In the work of indoor boundary extraction, almost all methods processed the point cloud by projecting them onto the XOY plane and then extending it to three dimensions using several techniques [15,16]. The techniques of extracting indoor boundary information by projection can be roughly divided into two categories: one is to obtain room boundaries by projecting point cloud onto a horizontal plane, which can effectively avoid the influence of indoor occluding objects [17,18]. However, there are usually openings such as doors and windows in the house boundary area, and the data dispersed from the openings during the scanning process will lead to local boundaries being different from the real situation to a large extent.

The other is to form the indoor boundary by directly projecting the point cloud of wall surfaces [19,20,21], which effectively avoid the influence of the dispersed point cloud in the boundary area. However, influenced by the occluded objects, all wall surfaces will not be completely obtained. Ochmann et al. [7] generated planar partition maps by extending the centerline of all candidate walls, and then assigned all partitioned planes to rooms or external areas. After determining a suitable label, connected components of identically labeled cells are rooms, and edges separating differently labeled regions are wall elements. However, this method is subject to local density of point cloud, resulting in room boundary information being incorrectly labeled. Nikoohemat et al. [21] complemented the wall surface by extending the local plane to the intersection line, but their method is only applicable to the locally missing wall surfaces. Bassier et al. presented four types of wall segment connections, which were obtained by user specification or data training to obtain the best wall segment connection [22]. Lim et al. [23] addressed the problem of obscured wall information by projecting non-building points onto a suitable building plane through scanner scan trajectories and poses.

Some studies segmented rooms by scanner trajectory information and viewpoint information, and their methods are effective not only for room segmentation, but also for architectural elements such as stairs, floors, and doors [24]. However, in practice, not all point clouds have scanner position and pose information. Therefore, some scholars study the problem of indoor room segmentation without scanner position and pose. Most of these studies project the point cloud to the XOY plane to form a floor plan, and then floor plans can be segmented by some methods, such as morphological methods [25]. Unlike methods based on floor plan segmentation, some methods divide the interior environment through space division and unit labeling methods. The space is first decomposed into cells by means of extracted wall lines [26], and then the label of each cell is confirmed by an energy minimization method. However, such an operation can lead to over-segmentation of rooms, especially for long corridors.

Door extraction has also received a lot of attention in recent years. Since the indoor environment contains many obscured objects, resulting in a large number of missing walls, it is of great nuisance to perform opening detection. Xiong et al. [2] introduced an approach to identify wall occlusions and opening detection. The approach marks the occluded part by ray tracing and then identifies the openings by support vector machine. Michailidis et al. [27] formulated the wall features extraction as a graph-cut optimization problem on a 2D cell complex defined by the line features of the reconstructed wall surface. Similar to the approach of Xiong et al., this study also required the location of the laser scanner as a prerequisite for complementation. Other than that, Kang et al. [10] used the point cloud with color information to identify closed and half-open doors. To collect point cloud with color information, an extra camera is needed and calibration between the camera and lidar is required.

In addition to this, some related studies working on indoor map construction and context labeling for indoor localization and navigation are ongoing. In terms of constructing indoor maps, Li et al. [28] presented a method to generate indoor floor plans based on WI-FI fingerprints. Similarly, Zhou et al. [29] proposed the new ray-aided generative adversarial model to automatically construct the radio map, which is used for the indoor WLAN intelligent target intrusion sensing and localization. Han et al. [30] constructed the indoor navigation network based on floor plans and walking habits. In terms of context labeling, some work on semantic segmentation of indoor environments is developing [31,32,33,34]. Yang et al. [10] construct indoor navigation networks based on indoor semantic information.

From the above analysis, there have been many research results on information extraction about navigation elements, but most of the methods have limitations in obtaining indoor navigation element information.

• For boundary extraction, the outlines of point cloud horizontal projection can be affected by the divergence data at the openings. The wall segments obtained from the wall projection are usually not able to provide the complete boundary outline information due to occlusions. Therefore, a single method usually does not provide good boundary due to the occlusions and local divergence data.
• For room segmentation, multiple building elements on multiple floors can be effectively segmented based on trajectory lines and viewpoints, but this method usually fails when faced with data without trajectory lines and viewpoints. The existing segmentation methods that do not rely on trajectory or viewpoint information suffer from under-segmentation or over-segmentation. The connecting transition space between rooms, as an important element of the indoor space, is rarely discussed in the literature.
• For door extraction, in cluttered indoor environments, the current method usually uses viewpoint information to project the occlusions in the room onto the wall to restore walls. In the absence of viewpoint information, walls that are heavily influenced by occlusions are usually unable to accurately extract door information. Secondly, there are some rooms with closed doors during data collection. Furthermore, it is still a challenge to detect closed doors in the point cloud without color information.

3. Methodology

3.1. Overview

In general, indoor space can be classified in two different applications: management of building components and indoor facilities; and use of indoor spaces [35]. Therefore, indoor space can be defined as a set of structural units with different functions composed of building components. As shown in Figure 1, according to the usage function and indoor navigation requirements, we divide the indoor space into navigable elements such as rooms, corridors and doors. Rooms and corridors are units separated by walls or partitions and enclosed by walls, ceilings and floors. Doors are the units responsible for connecting the various navigation elements in indoor navigation, and can be divided into “thin doors” and “thick doors” according to their characteristics [10,35]. The “thin door” is a two-dimensional plane embedded in an opening in the indoor wall surface. The “thick door” is a three-dimensional space enclosed by a connecting transition space and two “thin doors”, as shown in Figure 2. The connecting transition spaces are located inside the walls between rooms or corridors that have a connecting relationship.

According to the hierarchical relationship between building components and navigation elements, this paper gives the flowchart of information extraction and reconstruction of indoor navigation elements, as shown in Figure 3. Firstly, we extract the building components from the point cloud. Then, the information of indoor navigation elements is produced based on the building components. Finally, the indoor navigation elements and navigation network are reconstructed and represented using the building components. There are four main steps:

• Building components extraction. Extracting the possible planar data from the point cloud as alternative building components $\mathit{P}$ using the random sample consensus (RANSAC) algorithm [36]. Then the alternative building components are classified and integrated according to their space characteristics to obtain the walls, floors and ceilings;
• Wall segment restoration. Project the point cloud of the wall surface onto the XOY plane to form the wall segment raster. Then the connectivity of disjoint wall segments is judged according to the relationship between building components. Finally, the energy minimization function is constructed to determine the best connectivity;
• Navigation element information extraction. Diving building components into navigation elements such as rooms, corridors and connecting transition space based on their geometric features and topological relationships. Then the openings on the wall surface that satisfy the threshold value are extracted as doors. Finally, the navigation elements are optimized according to the relationship between doors and connecting transition space;
• Navigation element reconstruction and navigation network generation. First, reconstruct the navigation element 3D model using the building components and the extracted navigation element information. Then, the indoor navigation network is generated and represented based on the connectivity and local geometric features between the navigation elements.

Figure 3. Flowchart for reconstruction of navigation elements and network.

Figure 3. Flowchart for reconstruction of navigation elements and network.

3.2. Building Components Extraction and Wall Segment Restoration

We assume that the wall is perpendicular to the XOY plane. The plane, which is almost perpendicular to the XOY plane and with error less than ${\theta }_v$, is extracted from the alternative building components $\mathit{P}$ as the alternative wall surfaces ${\mathit{P}}_{\mathit{wall}}$. Affected by indoor occlusions and pedestrians, alternative wall surfaces include not only occluded wall surfaces, but also occlusions that may be considered as wall surfaces. We deal with occlusions as follows:

1. Occlusions treatment. First, the wall surfaces satisfying Equation (1) are combined, and then the wall surfaces smaller than the height of the door frame or with an area of less than 1 m² are excluded [10,37]. However, there may still be some higher occlusions that have not been removed. An optional step can be used to remove higher occlusions. If the height difference between the top of alternative wall surface and the ceiling satisfies Equation (2), the wall surface will be excluded. The point cloud of ceiling and floor can be obtained by the histogram method [38].

$$P_j=\left\{p_j\right|p_i,p_j\in {\mathit{P}}_{\mathit{wall}}:(ang\left(p_i,p_j\right)<{\theta }_b)\wedge (dis\left(p_i,p_j\right)<d)\wedge (h_i-h_j>h_p)\wedge \left(B_i{\displaystyle \cap }{B}_j\ne \varnothing \right)\}$$

(1)

where $ang$ and $dis$ represent the angle and the Euclidean distance between the planes. As shown in Figure 4, $h_i$ and $h_j$ are the height of the top of planes $p_i$ and $p_j$, $B_i$ and $B_j$ are the oriented bounding box of planes $p_i$ and $p_j$ after projection to the same plane. ${\theta }_b$ is the angle threshold, $d$ is the distance threshold, and $h_p$ is the height difference threshold between the top of the planes.

$$P_i=\{p_i\left|p_i\in {\mathit{P}}_{\mathit{wall}}:\right|h_{ceiling}-h_i|>h_p\}$$

(2)

where $h_{ceiling}$ represents the height of the ceiling, respectively, and $h_i$ represents the height of the alternative wall surface $p_i$;

2. Wall segment restoration. The detected wall surface is projected onto the XOY plane using a bottom-up method to generate a 2D wall segment evidence raster. If there are no points in the z-direction within the threshold, then no pixels will be projected on the 2D plane (the threshold is chosen to be the lowest door frame height) [39]. However, not all of the projected wall segments intersect with each other, and these disjoint wall segments should be restored to reconstruct a topologically correct 3D model [40]. The extraction and restoration process of the wall segments is shown in Figure 5.

For the processing flow described in Figure 5, we focus on topology analysis and wall segment restoration. The wall segments can be divided into boundary wall segments and indoor wall segments according to their locations. The boundary wall segments are located at the boundary between the indoor space and the external space. The indoor wall segments are located in the indoor space and are used to cut different structural units. According to the geometric and topological relationships between building elements, it is known that the projection of the outline of free-space evidence raster on the XOY plane coincides with the projection of the boundary wall surfaces, and the adjacent wall surfaces should have the same or adjacent ceilings. Thus, the outlines of the free-space evidence raster are obtained to guide the boundary wall segments for topological analysis. Then, the topological relationships of indoor wall segments are analyzed according to the spatial location of wall segments and ceilings. Algorithm 1 introduces the specific process.

Algorithm 1. Wall segment topology analysis

Input: Wall segment raster: Wall    Guide line (The outline of a free-space evidence raster): G    Wall segment search radius: rw    Connection search radius: rc    Ceiling: C

Output: Topological relations of wall segment: Tboundary (1) W = getDisjointLines (Wall); // Get disjoint wall segments in all wall segments (2) for each w$\in$W (3)   [Pcenter, Pend] = getMidPixelandEndPixel (w); (4) e = getForwardDirection (Pcenter, Pend); // Construct the forward direction from Pcenter to Pend (5) P = searchClosestPixel (Pend, G, rw); // Search for the nearest pixel in G to Pend within the radius rw (6) if (P == $\varnothing$) (7) Windoor.push_back(w); (8) continue; (9) end if (10) wtarget$\leftarrow \varnothing$ (11) while ((wtarget == $\varnothing$) && (wtarget != w)) (12) // Obtain pixels located around P with an angle of less than 90 degrees with $\mathit{e}$ from G (13) Ptemp = getForwardPixel(G, $\mathit{e}, \mathit{P}$); (14) $\mathit{e}$ = getForwardDirection (P, Ptemp); // Get the forward direction from P to Ptemp (15) P$\leftarrow$Ptemp; (16) wtarget = searchWallSegment (P, rw); // Search for wall segments around P within a radius of rw (17) end while (18) Tboundary.push_back ([w, wtarget]); (19) end for (20) for each w$\in$Windoor (21) c = getCeiling (w, C); // Get the ceiling associated with the wall segment (22) Pindoor = getEndPixel (w); // Get the endpoint of the wall (23) // Get the wall segment that is closest to the Pindoor and has the same or adjacent ceiling as w. (24) wtarget = getWallSegments (rc, Pindoor, c, w, Wall); (25) Tboundary.push_back ([w, wtarget]); (26) end for

After obtaining the topological relationships between the disjoint wall segments,, we determine whether the wall segments in the T_boundary are parallel to each other based on the angle threshold ${\theta }_b$. For non-parallel wall segments, three potential connection schemes are given, i.e., intersecting, orthogonal and directly connected, as shown in Figure 6a. For parallel wall segments, if the distance between the wall segments is less than the distance threshold $d$, the wall segments are combined; if the distance between the wall segments is greater than the distance threshold $d$, two potential connection schemes are given, i.e., orthogonal and directly connected., as shown in Figure 6b.

Figure 6 divides the local XOY plane into different cells by real and potential wall segments. These cells can be given different labels according to their location: indoor “$l_{int}$” and outdoor “$l_{ext}$”. The connection scheme is finally determined by the common edge between the indoor and outdoor cells. The labeling process of the cells can be solved similar to an energy minimization problem [41]. Our minimized energy function consists of data and smoothing terms:

$$E=\mathsf{\lambda }\sum _{c\in V}{D}_c\left(l_c\right)+\left(1-\mathsf{\lambda }\right)\sum _{c,c^{\prime }\in N}{V}_{c{c}^{\prime }}\left(l_c,l_{c^{\prime }}\right)$$

(3)

where $\sum _{c\in V}{D}_c\left(l_c\right)$ and $\sum _{c,c^{\prime }\in N}{V}_{c{c}^{\prime }}\left(l_c,l_{c^{\prime }}\right)$ represent the data term and the smoothing term. λ represents the balance parameter between the data term and the smoothing term.

The data items are determined by the proportion of each cell that contains free-space evidence raster.

$$D_c\left(l_c\right)=\{\begin{array}{cc}\frac{S_{V_c}\in occupied}{S_{V_c}}& l_c\in l_{int}\\ \frac{S_{V_c}\in empty}{S_{V_c}}& l_c\in l_{ext}\end{array}$$

(4)

where $S_{V_c}$ represents the number of pixels in the cell.$S_{V_c}\in occupied$ represents the number of pixels occupied by the free-space evidence raster in the cell.$S_{V_c}\in empty$ represents the number of pixels occupied by the empty raster in the cell.

Meanwhile, the smoothing term is determined by the pixel occupancy difference $P_{dif}$ of the free space-evidence raster between adjacent cells and the weight L(φ) of the common edges of adjacent cells.

$$\begin{array}{c}{V}_{c{c}^{\prime }}\left(l_c,l_{c^{\prime }}\right)=P_{dif}·L\left(\phi \right)+P_{dif}\\ P_{dif}=1-\left|\frac{S_{V_c}\in occupied}{S_{V_c}}-\frac{S_{V_c{}^{\prime }}\in occupied}{S_{V_c{}^{\prime }}}\right|\\ L\left(\phi \right)=\{\begin{array}{l}0, w_{c{c}^{\prime }}\in intersecting connection\\ \frac{1}{2}, w_{c{c}^{\prime }}\in direct connection\\ \frac{1}{2}, w_{c{c}^{\prime }}\in orthogonal connection AND \left(W_i\parallel W_j\right)\\ 1, w_{c{c}^{\prime }}\in orthogonal connection AND \left(W_i\nparallel W_j\right)\end{array}\end{array}$$

(5)

where $S_{V_c}$ and $S_{V_c{}^{\prime }}$ represent the number of pixels in two adjacent cells. $w_{c{c}^{\prime }}$ represents the connection relation of common edges of adjacent cells. Generally speaking, the fewer newly added edges in a potential scheme, the more likely it is to become a connected scheme. As shown in Figure 6, the intersection scheme has no new wall segment added. The direct connection scheme adds a new wall segment. The orthogonal scheme adds 1–2 new wall segments. The optimization problem defined in Equation (3) is solved by the method of graph cuts [42]. Then, the wall segments are restored according to the labels of the cells.

3.3. Indoor Room Segmentation

We propose a method that simulates the work of laser scanner to segment indoor space. The specific segmentation process is as follows:

Step 1: The restored wall segment projection raster and the free-space evidence raster are fused;

Step 2: Select a random point within the free-space evidence raster as the laser scanner site location L;

Step 3: Simulate the laser scanner at point L to emit n rays uniformly clockwise and record the wall segments hit by the rays;

Step 4: Extend the wall segment that was hit by the ray to form a plurality of virtual enclosed areas. Then, the enclosed area where L is located is used as the area to be processed in the subsequent steps;

Step 5: The proportion ∂ of real wall segments in the boundary of the virtual enclosed area is calculated according to Equation (6). If the proportion ∂ exceeds the threshold ${\epsilon }_{\partial }$, the virtual enclosed area is labeled as the area where the room or corridor is located. Instead, the area is discarded and step 2 is executed;

Step 6: Check whether the virtual enclosed area in step 4 has an adjacency or containing relationship with the already labeled area. If so, the area will be fused with the existing area and labeled as the same area;

Step 7: If the proportion of the labeled area to the free-space evidence raster exceeds α, the segmentation process is complete. The unmarked area is the connecting transition space.

$$\partial =\frac{W_r{\displaystyle \cap }{W}_v}{W_v}$$

(6)

where $W_r$ represents the real wall and $W_v$ represents the boundary of the virtual enclosed area.

As shown in Figure 7, when scanning within a room, the scanner can usually obtain a larger amount of wall surface data. In this case, it is sufficient to form a virtually enclosed area by adding a small number of wall segment extensions. Whereas, if the scanner is located in a connecting transition space between rooms, it will only obtain less wall surface data to segment room.

3.4. Door Extraction

Most existing methods for extracting indoor doors based on point clouds are projecting the point cloud onto a 2D raster and identifying the openings [2,27,43]. Since the walls are affected by occlusion and the density of the point cloud, different openings projected onto the 2D raster may be identified as the same opening by the connection between pixels. In the projected two-dimensional raster, the openings as doors are usually significantly different in length and width from the other openings caused by occlusion. Thus, we used a method based on statistical graphs of wall projections to fill the missing wall surfaces and extract doors using given thresholds. The method can be divided into four steps:

Step 1: The point cloud on the wall surface is projected onto a two-dimensional raster parallel to the wall surface and perpendicular to the XOY plane, as shown in Figure 8a. Then, noise is removed using a morphological open operation, as shown in Figure 8b;

Step 2: The black pixels in each column in Figure 8b are counted and the proportion of black pixels in the columns of the raster is calculated to generate a statistical graph, as shown in Figure 9a. The pixels at the peak of each column are smaller than the threshold $P_{col}$ that are filled as wall surface, as shown in Figure 8c;

Step 3: In Figure 8c, columns that have black pixels are continuously above the threshold $P_{col}$ and are recognized as local regions in turn. Then, the black pixels in each local region are separately counted by rows and the proportion of black pixels in the rows of the local region is calculated to generate statistical graphs, as shown in Figure 9b. For each local region, the rows above the threshold $P_{row}$ are filled as the wall surface and the rows less than the threshold $P_{row}$ are filled as black pixels, as shown in Figure 8d;

Step 4: The openings from the processed wall surface that meet the length and width thresholds are selected as doors, as shown in Figure 8e.

Figure 8. Door extraction process. Red represents the wall surfaces. Black represents the openings. Blue represents the alternative results. Green represents the extraction results. (a) is the 2D raster after point cloud projection on the wall surface. (b) is the result of (a) after morphological open operation. (c) is the result after some of the column pixels in (b) are filled. (d) is the result after some of the row pixels in (c) are filled. (e) is the result of the door extraction.

Figure 8. Door extraction process. Red represents the wall surfaces. Black represents the openings. Blue represents the alternative results. Green represents the extraction results. (a) is the 2D raster after point cloud projection on the wall surface. (b) is the result of (a) after morphological open operation. (c) is the result after some of the column pixels in (b) are filled. (d) is the result after some of the row pixels in (c) are filled. (e) is the result of the door extraction.

Figure 9. Statistics graph of wall projection raster. (a) represents the column statistics graph in step 2 of Figure 8b represents the row statistics graph of the purple boxed area in Figure 8b. The red line indicates the threshold values. (b) represents the column statistics graph in step 3 of Figure 8c.

Figure 9. Statistics graph of wall projection raster. (a) represents the column statistics graph in step 2 of Figure 8b represents the row statistics graph of the purple boxed area in Figure 8b. The red line indicates the threshold values. (b) represents the column statistics graph in step 3 of Figure 8c.

After the doors and connection transition spaces are extracted, there are still some “hidden doors” and connection transition spaces that need to be identified. Since some doors are closed during the data collection process, the location of the door may be considered as wall surface in door extraction. Secondly, since point cloud acquired by some laser scanning devices may have local drift. These drifted point clouds will be considered as connecting transition spaces when they appear inside the walls. To solve this problem, the following reasonable rules were made to automatically correct partially undetected doors and incorrectly detected connecting transition spaces. Rule 1 is used to detect “hidden doors”. Rule 2 is used as an optional rule to remove connection transition areas that are incorrectly constructed due to drifting point cloud.

Rule 1: If a “thin door” exists at the location where the connecting transition space connects to a structural unit, another “thin door” also exists at the location where the connecting transition space connects to another structural unit.

Rule 2 (Optional): If there are no “thin doors” on both side of the connecting transition space to the structural unit, the connecting transition space should not exist.

3.5. Navigation Network Generation and Model Reconstruction

After obtaining the information of building components and navigation elements in the indoor space, building components can be integrated based on the hierarchical relationship in Section 3.1 to reconstruct the navigation element models. The reconstructed results can generate not only models with geometric information, but also vector 3D models with semantic information to satisfy the applications of GIS and BIM.

For indoor navigation and route planning, we set the geometric center of the navigation element as a node and the connection relationship obtained from the label information in the connecting transition space or on both sides of the door as an edge to construct the navigation network, as shown in Figure 10a. However, navigation elements connected by only one edge may have a navigation path through the wall [30]. Therefore, some geometric features of indoor spaces need to be incorporated into the construction of navigation networks.

The concavity of the corner point of the navigation element is judged. If the angle of the corner point on the indoor side is ∂ (180° ≤ ∂ < 360°), the corner point is a concave point. When two wall segments on either side of the concave point intersect with the navigation network, these two wall segments are extended to intersect with other wall segments. Then, the local navigation network is reconstructed based on the result of segmentation, as shown in Figure 11. Besides, the structural units connected with more than three other units are set as corridors. As shown in Figure 10b, the navigation network of the corridor is also reconstructed according to the literature [10].

4. Experiments and Results

4.1. Experimental Data and Evaluation Criteria

The feasibility of the proposed method was verified by testing seven datasets using C++ and MATLAB on a computer (i7-11700 CPU, 2.5 GHZ and 16 GB RAM), as shown in Figure 12. The Synthetic Point Cloud (Synth) is an indoor environmental simulation data from the datasets of the University of Zurich. Synth includes three irregular rooms and three doors. The real data includes two offices and one room. The first office (Office1) was scanned using a Trimble TX5 terrestrial laser scanner at the Beijing University of Civil Engineering and Architecture. Office1 consists of a long corridor, six rooms and 18 doors. The second office (Office2) is also from the University of Zurich datasets. Office2 consists of a corridor, five rooms and six doors. Another room (Room) is a private residence scanned using a mobile measurement system with a laser scanner (Hokuyo UTM-30LX) [44]. Room consists of six rooms and four doors. In addition, we also selected TUB1, TUB2 second floor and UOM data from the ISPRS Benchmark Dataset. TUB1 data was captured using the Viametris iMS3D system with some missing wall boundaries. TUB1 includes 10 rooms and 23 doors. TUB2 data was captured using a Zeb-Revo sensor with low clutter. The second level of TUB2 includes 10 rooms and 28 doors. UOM data was captured using a Zeb-1 sensor acquired with a moderate level of clutter. UOM includes seven rooms and 14 doors. Three data sets in the Benchmark Dataset contain closed doors.

In order to verify the experimental results, the reconstructed model is evaluated by the following three criteria: completeness; correctness; and accuracy. Completeness is defined as the proportion of reference data in the source data, calculated by Equation (7). Correctness is defined as the proportion of the source data in the reference data, calculated by Equation (8). Accuracy is defined as the geometric proximity of the source data to the reference data and is calculated by Equation (9).

$$Completeness=\frac{Reference{\displaystyle \cap }Source}{Reference}$$

(7)

$$Correctness=\frac{Reference{\displaystyle \cap }Source}{Source}$$

(8)

$$Accuracy=\left|Distance\left(V_i,S_i\right)\right|$$

(9)

where $Source$ represents the source data, i.e., the door or 2D floor plane of the reconstructed model; $Reference$ represents the reference data, i.e., the door or 2D floor plane of the original point cloud or ground truth model. $Distance$ represents the Euclidean distance between data.$V_i$ represents the point on the surface of the reconstructed model. $S_i$ represents the closest point to $V_i$ on the source data (point cloud/ground truth model).

4.2. Parameter Settings

Table 1. Summary of parameters for model extraction and reconstruction.

	Parameter	Description	Units
Building component extraction	${\theta }_v$	Angle between alternative building components and alternative wall plane	Degree
	${\theta }_b$	Angle between alternative wall surfaces	Degree
	$d$	Distance between alternative wall surfaces	Meter
	$h_p$	Height difference between the top of alternative wall surfaces	Meter
Wall segment restoration	$r_w$	Radius of mutual search of boundary wall segments and guide lines	Meter
	$r_c$	Search radius of connection relations of indoor wall segments	Meter
	$\mathsf{\lambda }$	The balance parameter between the data term and the smoothing term	-
Indoor space decomposition	$n$	Number of emitted rays	-
	${\epsilon }_{\partial }$	Percentage of real wall segments in the boundary of the virtual enclosed area	-
	$\mathsf{\alpha }$	The percentage of the marked area in the free-space evidence raster	-
Door extraction	$h_{max}$	The maximum limit of door height	Meter
	$h_{min}$	The bottom limit of door height	Meter
	$w_{max}$	The maximum limit of door width	Meter
	$w_{min}$	The minimum limit of door width	Meter
	$P_{col}$	Filling threshold for column pixels on the wall surface	-
	$P_{row}$	Filling threshold for row pixels on the wall surface	-

shows the parameters used in this paper for building component extraction and model reconstruction. Four parameters are involved in the building component extraction. ${\theta }_v$ is recommended to be 0–5°. ${\theta }_b$, $d$ and $h_p$ are used to merge the alternative planes. ${\theta }_b$ setting range is recommended to be less than 20°. $d$ setting distance should be less than the minimum thickness of walls. $h_p$ is recommended to be greater than the height difference between the top of the highest occlusion and ceiling.

Three parameters are involved in the process of wall segment restoration. $r_w$ should be less than the minimum thickness of walls. $r_c$ should be greater than one and less than two times of the door width. λ is taken as 0.4–0.6 in our experiments. As shown in the local details in Figure 13, with increasing smoothing terms (i.e., decreasing λ), the local details will be ignored. Similarly, as the data items increase (i.e., λ increases), more details that match the existing data will be revealed. Thus, if the floor and ceiling point cloud is complete, λ should be increased when restoring the wall segment. Similarly, if the ceiling and floor point cloud is missing in localized areas, λ should be decreased. The room segmentation process involves three parameters. $\mathrm{n}$ should increase with the complexity of the room layout construction. ${\epsilon }_{\partial }$ is taken as 0.4–0.8. α should be slightly less than (1-area of connecting transition space /area of indoor space).

Door extraction process involves six parameters. The door height limit and width limit should be obtained according to the actual situation. $P_{col}$ and $P_{row}$ can be a unitized scale (e.g., 0.5). The method of determining $P_{col}$ is based on the height of the wall where the door is located, the height of the door, the height of the occlusion inside the door and the height of other openings at the location of the door. $P_{col}$ can be calculated by Equation (10). Similar to $P_{col}$, the method of determining $P_{row}$ is based on the width of the door and the width of the occlusion inside the door. $P_{row}$ can be calculated by Equation (11).

$$P_{col}=\frac{h_{door}-h_{occ}+h_{other}}{h_{wall}}$$

(10)

$$P_{row}=\frac{w_{door}-w_{occ}}{w_{door}}$$

(11)

where $h_{door}$ represents the height of the door, $h_{occ}$ represents the height of the occlusion, $h_{other}$ represents the height of other openings at the location of the door, $h_{wall}$ represents the height of the wall surface, $w_{door}$ represents the width of the door and $w_{occ}$ represents the width of the occlusion. As shown in Figure 14.

4.3. Experimental Results

Table 2. Model reconstruction and navigation network generation results.

Data	Indoor Space Decomposition	Navigation Network	Model (Hide Ceiling)
Synth
Office1
Office2
Room
TUB1
TUB2
UOM

shows the reconstructed model and the navigation network, where the model has semantic information not only for the ceiling, floor and walls, but also for the doors and connecting transition space.

5. Discussion

5.1. Model Accuracy Evaluation

To evaluate the accuracy of the model, the reconstructed results of three data from the ISPRS benchmark dataset were compared with ground truth models. The evaluation used Equation (9) to calculate the unsigned deviation between planes. Figure 15 shows the unsigned distance deviations using colors and plots the statistical histogram. Among them, the deviation in TUB1 data is mainly distributed in the range of 0–5 cm, the mean of distance deviation is about 1.3 cm, the root mean square (RMS) of distance deviation is about 2.2 cm. The deviations in TUB2 data were concentrated in the range of 0–10 cm, the mean of distance deviation was about 3.1 cm, the RMS of distance deviation was about 6.3 cm. The deviation in UOM data is concentrated in the range of 0–9 cm, the mean of distance deviation is about 4.3 cm, the RMS of distance deviation was about 7.3 cm. Compared with other methods submitted to the ISPRS indoor modeling benchmark such as [45], our methods have a higher reconstruction accuracy for TUB1 and TUB2. Meanwhile, the accuracy of the reconstructed models for the UOM dataset is slightly lower.

Since the other four datasets do not have real ground truth models, the unsigned distance deviation between the reconstructed models and the original point cloud is calculated for quantitative evaluation. Figure 16 shows the unsigned distance deviations using colors and plots the statistical histogram. Among them, the deviations in Synth are concentrated between 0–3 cm, the mean of distance deviation is about 2.1 cm, the RMS of distance deviation was about 4.8 cm. The deviations in the data Room are concentrated in the range of 0–10 cm, the mean of distance deviation is about 3.4 cm, the RMS of distance deviation was about 7.2 cm. The deviations in the data Office1 are concentrated in the distribution of 0–10 cm, the mean of distance deviation is about 5.2 cm, the RMS of distance deviation was about 9.7 cm. The deviations in the data Office2 are concentrated in the range of 0–5 cm, the mean of distance deviation is about 3.9 cm, the RMS of distance deviation was about 7.5 cm.

5.2. Room Sgementation Evaluation

We compare the experimental results with the relevant segmentation methods using the same dataset, and the results are shown in

Table 3. Result comparison of room segmentation methods.

Data	TUB1	Office1	Room
Proposed segmentation method
Other segmentation method
	Ambru, et al.	Morphology-based segmentation	Yang et al.

. The method of Ambru [18] et al. assigns different labeling information to each individual segmented region after segmenting the ceiling projection by an extended wall line, however, some regions may be assigned wrong labels due to the local ceiling missing. The morphology-based segmentation method is the more classic segmentation scheme among room segmentation methods, although it cannot clearly delineate the boundaries of rooms. The method proposed by Yang [38] et al. can better divide the indoor space, but for the connecting transition space between rooms, which are directly classified as indoor space or eliminated. Compared with the above three segmentation schemes, the segmentation method in this paper can effectively segment the indoor space and has the following advantages: (i) our room segmentation method can effectively avoid the segmentation error caused by the missing local ceiling; (ii) The segmentation accuracy of each room boundary is ensured; (iii) The connecting transition space between rooms can be well extracted.

5.3. Component Evaluation

As shown in

Table 4. Room reconstruction evaluation results of benchmark dataset.

Data	Real Room Number	Detected Real Room Number	Correctness on 2D Floor Plane	Completeness on 2D Floor Plane
TUB1	10	11	0.987	0.992
TUB2	10	9	0.981	0.952
UOM	7	7	0.983	0.936

, by comparing the experimental results of the three datasets provided by the ISPRS benchmark with the terrestrial truth model. The data TUB1 detected 10 rooms with 98.7% correctness and 99.2% completeness of the 2D floor planes. The data TUB2 detected seven rooms with 98.1% correctness and 95.2% completeness of the 2D floor plan. The data UOM detected seven rooms with 98.3% correctness and 93.6% completeness of the 2D floor plan.

Table 5. Opening for benchmark data.

Data	Door Number	Detected Door Number	Correctly Detected Door Number	Correctness on Door	Completeness on Door
TUB1	23	22	20	0.91	0.87
TUB2	28	29	27	0.93	0.96
UOM	14	13	12	0.92	0.86

gives the results of the door detection and room connectivity assessment, and the correctness and completeness of the room door detection was greater than 86% in all rooms.

For the remaining four groups of indoor environments without real surface data, the generated models are compared with the point cloud to analyze and evaluate the navigation element extraction results. As shown in

Table 6. Experimental results of some experiments without real ground data.

Data	Real Room Number	Detected Real Room Number		Detected Wall Surface Number
Synth	3	3		25
Office1	7	7		106
Office2	6	6		36
Room	6	6		29
Data	Door Number	Detected Door Number	Correctnesson Door	Completenesson Door
Synth	3	3	1	1
Office1	18	18	0.88	0.88
Office2	5	5	1	1
Room	4	5	0.80	1

, all four groups of experimental room segmentation results are the same as the correct results, and the correctness and completeness of the results of door detection is greater than 80%.

5.4. Limitations

In the above data, most of the information of walls, rooms and doors are detected, but there are some errors in the detection process. In the process of wall restoration, although several given connection schemes are able to restore the missing walls, these schemes are too limited to represent all connection situations due to the complicated building structure. In the door extraction, the “thick doors” with occlusion can be extracted well, but the “thin doors” with a lot of occlusion cannot be extracted in this paper, as shown in Figure 17. Other point cloud information (e.g., density, intensity, color, etc.) can be tried for extraction. Secondly, this paper uses the plane as the building element for navigation element extraction. The reconstruction results will be different from the actual situation when facing building components with larger curvature or rooms with fewer planes. In addition, the laser scanner cannot accurately acquire data on the surface of the object in some scenes due to the influence of the object surface material. Therefore, the method in this paper is also not applicable to indoor scenes where a large number of objects are used that affect the data acquisition by the laser scanner.

6. Conclusions and Future Work

This paper presents methods to automatically reconstruct indoor navigation elements from unstructured 3D point cloud of buildings with occlusions and openings. To reiterate the value of the proposed methods, we summarized the process taking a practical work example as shown in Figure 18. This is a real indoor scene consisting of a long corridor and six rooms. The indoor scene contains a lot of occlusion. We accurately reconstructed the navigation network and the 3D model. This is difficult for other methods to successfully extract some elements such as connecting transition spaces and “hidden doors”. The methods extract possible planes from the point cloud as building components. The walls are restored using the point cloud horizontal projection information. Next, the indoor space is segmented by simulating the work of the laser scanner. Then, we used the statistical graph to fill the missing walls and set thresholds to obtain information about the doors. The reconstructed building models were also optimized using the relationship between doors and connecting transition space. Finally, the indoor navigation element model and the navigation network are generated. The proposed methods were tested on different building datasets with complex structures and occlusions. The experiments show that the method can detect and reconstruct indoor navigation elements and navigation networks without viewpoint information. The mean deviation in the reconstructed model is in the range of 0–5 cm, and the completeness and correctness are greater than 80%. These results demonstrate the robustness and effectiveness of the method. However, the proposed method also has some limitations, for example, for “thin doors” with a large amount of occlusion, the method in this paper is not successful in extracting them and needs to be combined with other point cloud information (e.g., density, intensity, color, etc.). Secondly, the reconstruction results will be different from the actual situation when facing building components with larger curvature or rooms with fewer planes.

In the future, we intend to extend the existing approach to indoor environments containing building structures with large curvature. Facing the missing wall segments caused in cluttered environments, we will try to build a semantic model library to provide better restoration solutions for the missing wall segments. Finally, indoor path planning in cluttered environments is also the next direction we will investigate.