Figure 1.
Workflow of the proposed algorithm for building change detection from point cloud data and aerial images.
Figure 1.
Workflow of the proposed algorithm for building change detection from point cloud data and aerial images.
Figure 2.
Superposition of the co-segmentation and the gridded RGB images composed of RGB values of gridded color points at Time t1 and its enlarged subsets of the bi-temporal images. (a) Superposition of the co-segmentation and the gridded RGB image at Time t1, and the enlarged subsets of the bi-temporal modeled in the red rectangle; (b) Superposition of the co-segmentation and the gridded gray elevation image at Time t1, and the enlarged subsets of the bi-temporal gray elevation images in the red rectangle.
Figure 2.
Superposition of the co-segmentation and the gridded RGB images composed of RGB values of gridded color points at Time t1 and its enlarged subsets of the bi-temporal images. (a) Superposition of the co-segmentation and the gridded RGB image at Time t1, and the enlarged subsets of the bi-temporal modeled in the red rectangle; (b) Superposition of the co-segmentation and the gridded gray elevation image at Time t1, and the enlarged subsets of the bi-temporal gray elevation images in the red rectangle.
Figure 3.
An example of a grid point and its corresponding image points obtained by projection equation.
Figure 3.
An example of a grid point and its corresponding image points obtained by projection equation.
Figure 4.
The gridded RGB image and its corresponding building detection with semantic segmentation at Time t1. (a) The gridded RGB image; (b) Building detection with semantic segmentation.
Figure 4.
The gridded RGB image and its corresponding building detection with semantic segmentation at Time t1. (a) The gridded RGB image; (b) Building detection with semantic segmentation.
Figure 5.
Superposition of the building detection with semantic segmentation and the ground points from LiDAR-DSM, and its corresponding subsets of building detection and ground points.
Figure 5.
Superposition of the building detection with semantic segmentation and the ground points from LiDAR-DSM, and its corresponding subsets of building detection and ground points.
Figure 6.
Extraction of the changed buildings at Time t1 and t2. (a) Extraction of the changed buildings at Time t1; (b) Extraction of the changed buildings at Time t2.
Figure 6.
Extraction of the changed buildings at Time t1 and t2. (a) Extraction of the changed buildings at Time t1; (b) Extraction of the changed buildings at Time t2.
Figure 7.
Overview of the dataset 1 and its enlarged subsets. (a) Overview of the dataset 1 at Time t1 and its enlarged subsets; (b) Overview of the dataset 1 at Time t2 and its enlarged subsets.
Figure 7.
Overview of the dataset 1 and its enlarged subsets. (a) Overview of the dataset 1 at Time t1 and its enlarged subsets; (b) Overview of the dataset 1 at Time t2 and its enlarged subsets.
Figure 8.
Overview of dataset 2 and its enlarged subsets. (a) Time t1; (b) Time t2.
Figure 8.
Overview of dataset 2 and its enlarged subsets. (a) Time t1; (b) Time t2.
Figure 9.
Orthophotos, ground truth, semantic segmentation results, and result evaluations.
Figure 9.
Orthophotos, ground truth, semantic segmentation results, and result evaluations.
Figure 10.
Results and evaluations of the methods on the Nanning dataset. (a–d) are the results of semantic differencing (SemanticDIF), digital surface model difference (DSMDIF), Post-refinement, and the proposed method, respectively; (e) is the ground truth of the Nanning dataset; (f–i) are the result evaluations of SemanticDIF, DSMDIF, Post-refinement, and the proposed method, respectively.
Figure 10.
Results and evaluations of the methods on the Nanning dataset. (a–d) are the results of semantic differencing (SemanticDIF), digital surface model difference (DSMDIF), Post-refinement, and the proposed method, respectively; (e) is the ground truth of the Nanning dataset; (f–i) are the result evaluations of SemanticDIF, DSMDIF, Post-refinement, and the proposed method, respectively.
Figure 11.
Examples of FN and FP1 of dataset 1. (a) FN caused by semantic segmentation at Time t1 and t2; (b) FN caused by normalized digital surface models (nDSM) at Time t1 and t2; (c) FP1 caused by errors in digital surface models (DSM) at Time t1 and t2; (d) FP1 caused by small buildings at Time t1 and t2.
Figure 11.
Examples of FN and FP1 of dataset 1. (a) FN caused by semantic segmentation at Time t1 and t2; (b) FN caused by normalized digital surface models (nDSM) at Time t1 and t2; (c) FP1 caused by errors in digital surface models (DSM) at Time t1 and t2; (d) FP1 caused by small buildings at Time t1 and t2.
Figure 12.
Ground truth and results of the proposed method of dataset 2. (a) Ground truth; (b) Results of the proposed method from bi-temporal LiDAR-DSMs and aerial images.
Figure 12.
Ground truth and results of the proposed method of dataset 2. (a) Ground truth; (b) Results of the proposed method from bi-temporal LiDAR-DSMs and aerial images.
Figure 13.
Corresponding result evaluations of dataset 2 from LiDAR-DSM and dense image matching digital surface models (DIM-DSM). (a) Result evaluation from LiDAR-DSM and its enlarged subsets; (b) Result evaluation from DIM-DSM and its enlarged subsets.
Figure 13.
Corresponding result evaluations of dataset 2 from LiDAR-DSM and dense image matching digital surface models (DIM-DSM). (a) Result evaluation from LiDAR-DSM and its enlarged subsets; (b) Result evaluation from DIM-DSM and its enlarged subsets.
Figure 14.
Examples of FPs of dataset 2 from LiDAR-DSM. (a) FP caused by suspected building objects; (b) FP caused by penetrating objects; (c) FP caused by terrain change.
Figure 14.
Examples of FPs of dataset 2 from LiDAR-DSM. (a) FP caused by suspected building objects; (b) FP caused by penetrating objects; (c) FP caused by terrain change.
Figure 15.
Examples of FP1 and FN of dataset 2. (a) FP1 caused by broken roofs; (b) FN caused by small changed buildings sheltered by trees; (c) FN caused by confused changed buildings.
Figure 15.
Examples of FP1 and FN of dataset 2. (a) FP1 caused by broken roofs; (b) FN caused by small changed buildings sheltered by trees; (c) FN caused by confused changed buildings.
Figure 16.
Overview at Time t1 and enlarged subsets of difference of LiDAR-DSM and dense image matching digital surface models (DIM-DSM), and the corresponding orthophotos of dataset 2 at Time t1 and t2. (a) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area A at Time t1; (b) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area A at Time t2; (c) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area B at Time t1; (d) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area B at Time t2; (e) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area C at Time t1; (f) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area C at Time t2.
Figure 16.
Overview at Time t1 and enlarged subsets of difference of LiDAR-DSM and dense image matching digital surface models (DIM-DSM), and the corresponding orthophotos of dataset 2 at Time t1 and t2. (a) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area A at Time t1; (b) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area A at Time t2; (c) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area B at Time t1; (d) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area B at Time t2; (e) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area C at Time t1; (f) Enlarged subset of difference of LiDAR-DSM and DIM-DSM, and its corresponding orthophoto of area C at Time t2.
Figure 17.
Histograms of difference of LiDAR-DSM and dense image matching digital surface models (DIM-DSM) of dataset 2 at Time t1 and t2. (a) Histograms of difference of LiDAR-DSM and DIM-DSM of dataset 2 at Time t1; (b) Histograms of difference of LiDAR-DSM and DIM-DSM of dataset 2 at Time t2.
Figure 17.
Histograms of difference of LiDAR-DSM and dense image matching digital surface models (DIM-DSM) of dataset 2 at Time t1 and t2. (a) Histograms of difference of LiDAR-DSM and DIM-DSM of dataset 2 at Time t1; (b) Histograms of difference of LiDAR-DSM and DIM-DSM of dataset 2 at Time t2.
Figure 18.
Object-based statistics of building change detection with different parameters. (a) Object-based statistics of building change detection with and NS ranging from 0.1S to 2S; (b) Object-based statistics of building change detection with and NC ranging from 2 to 50; (c) Object-based statistics of building change detection with and NH ranging from 0.5 to 10; (d) Object-based statistics of building change detection with and ranging from 0.01 to 0.5; (e) Object-based statistics of building change detection with and ranging from 0.5 to 2.5.
Figure 18.
Object-based statistics of building change detection with different parameters. (a) Object-based statistics of building change detection with and NS ranging from 0.1S to 2S; (b) Object-based statistics of building change detection with and NC ranging from 2 to 50; (c) Object-based statistics of building change detection with and NH ranging from 0.5 to 10; (d) Object-based statistics of building change detection with and ranging from 0.01 to 0.5; (e) Object-based statistics of building change detection with and ranging from 0.5 to 2.5.
Table 1.
Change type determination with the guidance of a priori knowledge.
Table 1.
Change type determination with the guidance of a priori knowledge.
Time t1/Time t2 | Changed Building | No Building Change |
---|
Changed Building | Taller, if DSMt1 < DSMt2 Lower, if DSMt1 > DSMt2 | Demolished |
No building change | Newly built | No building change |
Table 2.
Details of the bi-temporal stereo aerial images of dataset 1.
Table 2.
Details of the bi-temporal stereo aerial images of dataset 1.
Dataset | Shooting Time | Camera | Focal Length | Image Size | Pixel Size | GSD | Number of Images | Forward Lap | Side Lap |
---|
t1 | 2012 | DMC | 120 mm | 7680 × 13,824 | 12 µm | 17 cm | 10 | 65% | 35% |
t2 | 2013 | DMC | 120 mm | 7680 × 13,824 | 12 µm | 17 cm | 10 | 65% | 35% |
Table 3.
Details of the bi-temporal stereo aerial images of dataset 2.
Table 3.
Details of the bi-temporal stereo aerial images of dataset 2.
Dataset | Shooting Time | Camera | Focal Length | Image Size | Pixel Size | GSD | Number of Images | Forward Lap |
---|
t1 | September 2011 | P65+ | 52 mm | 5989 × 4488 | 9 µm | 13 cm | 55 | 70% |
t2 | August 2012 | P65+ | 52 mm | 5989 × 4488 | 9 µm | 13 cm | 66 | 70% |
Table 4.
Confusion matrix with different change types, where yellow represents true positive (TP), blue represents false negative (FN), rose and brown represent two types of false positive (e.g., FP1 and FP), and white represents true negative (TN).
Table 4.
Confusion matrix with different change types, where yellow represents true positive (TP), blue represents false negative (FN), rose and brown represent two types of false positive (e.g., FP1 and FP), and white represents true negative (TN).
Proposed\Ground Truth | No Building Change | Newly Built | Taller | Demolished | Lower |
---|
No building change | N11 | N12 | N13 | N14 | N15 |
Newly built | N21 | N22 | N23 | N24 | N25 |
Taller | N31 | N32 | N33 | N34 | N35 |
Demolished | N41 | N42 | N43 | N44 | N45 |
Lower | N51 | N52 | N53 | N54 | N55 |
Table 5.
Pixel-based statistics of semantic segmentation of the two datasets.
Table 5.
Pixel-based statistics of semantic segmentation of the two datasets.
Datasets | Pixel-Based Statistics |
---|
Cr | Cm | Q |
---|
Nanning | 2011 | 73.8% | 87.2% | 97.7% |
2012 | 62.4% | 83.3% | 95.7% |
Guangzhou | 2012 | 82.1% | 92.4% | 93.2% |
2013 | 81.0% | 90.9% | 92.3% |
Table 6.
Statistics of the comparison over the Nanning dataset.
Table 6.
Statistics of the comparison over the Nanning dataset.
Methods | Pixel-Based Statistics | Object-Based Statistics |
---|
Cr | Cm | Q | Cr | Cm | Q |
---|
SemanticDIF | 37.0% | 62.8% | 94.6% | - | - | - |
DSMDIF | 17.9% | 86.0% | 84.8% | - | - | - |
Post-refinement | 61.2% | 62.6% | 97.6% | 88.6% | 98.0% | 86.9% |
Proposed | 64.1% | 64.0% | 97.8% | 92.9% | 96.8% | 90.1% |
Table 7.
Confusion matrix of the building change detection in Dataset 2.
Table 7.
Confusion matrix of the building change detection in Dataset 2.
Proposed\Ground Truth | No Building Change | Newly Built | Taller | Demolished | Lower |
---|
LiDAR-DSM+LiDAR-DTM+IMAGES | No building change | 0 | 11 | 1 | 9 | 0 |
Newly built | 12 | 101 | 9 | 0 | 0 |
Taller | 1 | 0 | 107 | 0 | 0 |
Demolished | 11 | 0 | 1 | 44 | 0 |
Lower | 1 | 0 | 0 | 0 | 2 |
Dense image matching digital surface models (DIM-DSM)+ DIM-DTM+ IMAGES | No building change | 0 | 19 | 4 | 13 | 0 |
Newly built | 52 | 90 | 6 | 0 | 0 |
Taller | 6 | 1 | 105 | 0 | 0 |
Demolished | 44 | 0 | 3 | 40 | 0 |
Lower | 7 | 0 | 0 | 0 | 2 |
Table 8.
Object-based statistics of the proposed building change detection method in the two datasets.
Table 8.
Object-based statistics of the proposed building change detection method in the two datasets.
Proposed\Ground Truth | Dataset 1 | Dataset 2 |
---|
Cr | Cm | Q | Cr | Cm | Q |
---|
LiDAR | LiDAR-DSM+ LiDAR-DTM+IMAGES | - | 87.9% | 92.4% | 81.9% |
LiDAR-DSM+IMAGES | - | 75.6% | 92.4% | 71.2% |
DIM | Dense image matching digital surface models (DIM-DSM)+ LiDAR-DTM+IMAGES | - | 70.7% | 89.8% | 65.4% |
DIM-DSM+DIM-DTM+IMAGES | 92.9% | 96.8% | 90.1% | 66.6% | 86.8% | 60.5% |
DIM-DSM+IMAGES | 82.4% | 96.7% | 80.2% | 59.8% | 89.7% | 56.0% |