With the emerging implementation of an unpiloted (or unmanned) aerial system (UAS) structure-from-motion (SfM) deployments for various field data collection, accuracy becomes a concern for engineering and survey applications and feature extraction. However, the introduction of ground control points (GCPs) and checkpoints (CPs) aims to improve accuracy and quantify the errors in the resultant point clouds [
1]. The surveyed sites investigated in this paper consist of diverse geometry and a range of elevations, including vegetation, an active construction project, paved and gravel roadways, geotechnical slopes, etc. These sites are selected given their diversity in the landscape and features in the scene of interest. The authors were equipped with two light detection and ranging (lidar) scanners and a UAS with an onboard camera for data collection, which allows for a detailed comparison between the data modalities. For many structures and civil infrastructure with limited accessibility, UAS is an efficient, accurate, and economical approach for data acquisition to produce a three-dimensional point cloud. A point cloud is a set of vertices in three-dimensional space that can be used for surveying, measurements, and structural assessments (e.g., [
2,
3,
4]). In contrast, lidar point cloud collection can be more time consuming, but it can be characteristic of high accuracy (at the centimeter level for registration). In this study, the UAS SfM errors are validated throughout the point cloud to locate and quantify their distribution.
Point cloud datasets are commonly collected for surveying and engineering applications. To perform this task efficiently, UAS based photogrammetric surveys are an optimal option [
5], particularly given their overhead view of the site and the availability of inaccessible locations [
6]. UAS data acquisition typically includes digital images and georeferencing information via a ground survey that can produce a point cloud using an advanced computer vision technique known as SfM. SfM uses a series of two-dimensional images with sufficient overlap to estimate the 3D reconstructed scene [
7]. Given its efficiency, accuracy, density, and lower-cost (as rotary wing UAS in comparison to fixed-wing piloted aircraft surveys), UAS point cloud data acquisition has been widely applied to the areas of engineering, transportation, geology, surveying, etc. A few examples include roadside feature detection and feature extraction (e.g., [
8]), landslide monitoring (e.g., [
9]), and detailed surveying data (e.g., [
10]).
One of the most popular deployments of UASs is following natural disasters for reconnaissance purposes. Natural disasters such as earthquakes, tornadoes, and hurricanes can cause many injuries, financial losses, and damage to civil infrastructure and agriculture (e.g., [
11,
12]). Following these events, post-disaster scene reconstruction is often limited by time and site accessibility imposed by precarious structures, debris, road closures, etc. However, rapid post-disaster assessments enable first responders and emergency managers logistical planning and effective deployment and damage assessment, loss estimates, and infrastructure assessment for insurance adjusters, engineers, and researchers. An example of post-hurricane site data collection via UAS is displayed in
Figure 1 [
13], in which the structure was in an extremely precarious state of potential collapse. For this example site and others with these similar limitations, SfM is a rapid, cost-effective, robust, and efficient approach for 3D point cloud data collection. Moreover, the operation ease and low-cost of UAS deployment also provide a comprehensive application for 3D modeling [
14], where a 3D model, in this case, refers to a point cloud. The collected dataset can be permanently preserved and the investigation using 3D modeling.
1.1. Literature Review
As an emerging technique, point clouds have been widely implemented in infrastructure assessments. However, traditional assessment approaches are still the most common, particularly in surveying, mapping, and post-disaster evaluation. Various methodologies and applications were introduced in past years to improve efficiency, accessibility, and accuracy and reduce the subjectivity. In what follows, a select literature review describes some of the recent studies that have used UASs for civil infrastructure assessments with a focus on the extensive applications and advantages.
In an early study, Adams et al. (2013) investigated the application of UASs in a post-disaster assessment at neighborhood and individual building scales following the 2012 Northern Alabama EF-3 tornado outbreak [
16]. The study presented a UAS survey of two severely damaged residential buildings as a case study and reported that the team was able to collect images with a ground sampling distance of 2 mm via very low above ground level (AGL) flights using the onboard 12-megapixel camera. In addition, Adams et al. (2013) were able to observe and identify roof damage and specific building material in the debris field and perform quantitative analyses after a stereo-photogrammetric reconstruction. A case study by Chiu et al. (2017) proposed UAS-SfM processing and application for larger structures [
17]. The case study focused on the deformation measurement of a 470 by 170-m membrane floating cover at the Melbourne Water Western Treatment Plant. Traditional structural health monitoring (SHM) techniques rely on surface mounted or built-in sensors, which can be a long-term and durable sensing method; however, it is often costly and dependent on sensor reliability. Given the efficiency, reliability, availability, and the low-cost of a UAS derived point cloud, UAS flights can be an optimal technique for data collection. In this study, three flight variations were carried out on individual dates to validate errors. The three flight operations included: (1) manual operation with UAS onboard GPS (typical consumer-grade accuracy around 5–10 m, (e.g., [
18,
19]) and without any RTK (real-time kinematic) ground control points; (2) flew autonomously via waypoints with a 50% imagery overlap with surveyed ground control points (GCPs); and (3) flew autonomously via waypoints with a 70% imagery overlap with ground control points (GCPs). As a result, the third flight yielded the best product in terms of accuracy, which was estimated at 0.46-m. In conclusion, the authors stated that this technology is also applicable to other high-value assets in SHM, as a novel approach to remotely inspect the inaccessible, large scale infrastructure. Note, however, that the level of error may or may not be acceptable for all projects as it is dependent on the type of survey and the deliverables requested.
UAS SfM data are also widely used in digital elevation models (DEMs) development (e.g., [
20,
21,
22,
23,
24]). For example, the UAS SfM application proposed by Thiebes et al. (2016) analyzed a UAS derived digital surface model (DSM) in comparison to an existing 2006 lidar DSM dataset [
24]. The study focused on a large landslide of an approximate volume of 30 million cubic meters. The two datasets were collected over nine years, where the UAS SfM data collection included 2033 images resulting in a GSD of 1.5 cm and a root mean square error (RMSE) of 6.3 cm. The temporal comparison was carried out by a DEM of difference and kinematics, where the maximum displacement reached 12 m. The authors concluded that the UAS-SfM dataset was useful on an annual basis, but the UAS survey requires careful interpretation for detailed analysis. Meanwhile, UAS-SfM applications are well documented in the literature. For example, UAS-SFM has been involved in various assessment applications, including roadways (e.g., [
25,
26,
27]), railroads (e.g., [
28,
29]), structures (e.g., [
5,
7,
30]), geotechnical slopes (e.g., [
31,
32]), and agricultural crops (e.g., [
33,
34,
35]), and deep learning-based structural damage classification following natural hazard events (e.g., [
36,
37]).
Cloud-to-cloud distance computations between two or more point clouds are very useful in engineering applications, particularly to compute temporal changes. One of the more common methods to compute these differences includes the multiscale model to model cloud comparison (M3C2). Lague et al. (2013) developed this methodology in the open-source CloudCompare platform to quantify the surface changes within complex topography [
38]. This work detailed the landscape and topographic changes at the Rangitikei River in New Zealand, including bedrock cliffs, riverbanks, rockfall debris, etc. The authors directly compared their M3C2 algorithm with existing methods (cloud-to-cloud and cloud-to-mesh). From their results, it was concluded that the M3C2 algorithm computes accurate surface changes and, more importantly, demonstrates its independence of factors like point density and surface roughness within the M3C2 algorithm.
Following the development of the M3C2 algorithm, Warrick et al. (2019) and Peppa et al. (2019) successfully carried out landslide quantifications using M3C2 on UAS-SfM point clouds [
39,
40]. In a unique dataset, Rossi et al. (2019) analyzed the change detection of an underwater coral reef using the M3C2 function [
41]. Another more recent study conducted by Nesbit and Hugenholtz (2019) investigated the accuracy of UAS SfM 3D models in landscapes with a direct comparison to reference terrestrial lidar scanner (TLS) data [
19]. The study site is a 100 by 80-m fossil-rich park with complex topography. The authors investigated the uncertainty caused by the image acquisition angle varying from 0 to 35°. The data processing revealed the need to combine various flight directions to reduce coverage inconsistencies and the required image overlap for sufficient point density. In a comparison of the processed SfM to the TLS point cloud, the SfM dataset underestimated high elevation points and overestimated low elevation points for their study. However, with the introduction of ground control points, the cloud-to-cloud distance measured in CloudCompare via M3C2 function was comparable at the centimeter level. The research did not, however, indicate an efficient number of ground control points to minimize the errors distributed in the point cloud.
Due to the high reliability and accuracy of a lidar point cloud, it is commonly used to compare UAS SfM results quantitatively. Lidar accuracy was discussed in Cawood et al. (2018) via a laboratory test to quantify the accuracy of static and mobile lidar scanning systems [
42]. In this study, the authors investigated the differences at target coordinates and cloud-to-cloud differences using the M3C2 function. The authors found subcentimeter differences at the target locations and an M3C2 mean distance of 0.57 cm, which indicated no significant offset between the two datasets. For a second example, Guisado-Pintado et al. (2019) demonstrated a comparison of TLS and UAS 3D mapping on a temperate beach-dune zone (with an area of 11,520 m
2) [
43]. The authors discussed the efficiency, associated challenges, and relative performance over various terrain types and analytical approaches. In conclusion, both datasets were useful for mapping with different vegetation coverage with a reported mean error of 6 cm. TLS surveys produced more realistic surface models, especially for sparsely vegetated areas. However, UAS SfM had a key advantage over TLS for its rapid surveying time and ease of operation [
44]. In another example, Nouwakpo et al. (2016) used UAS SfM and TLS to quantify soil surface microtopography. This was conducted on a 36 m
2 region of interest where the difference of DEMs evaluated the two data collection platforms. The elevation difference in their study was 3 ± 5 mm. This centimeter-level error was acceptable, and therefore, the authors recommend the UAS SfM approach due to its key advantage in field research in regard to efficient data collection time and minimal processing effort.
Specific to the analysis and monitoring of structures and infrastructure, in some emergency situations, surveys can be restrained by time and security [
2]. For example, Martínez-Espejo Zaragoza et al. (2017) outlines how TLS, sometimes referred to as ground-based lidar (GBL) [
45], and UAS platforms were selected for data acquisition. TLS and UAS SfM were used along with real-time kinematic (RTK) surveyed ground control points to obtain a 3D model of the area of interest. The two datasets were compared quantitatively, where the structure of interest was the wood-framed Harzburger Hof Hotel, four stories in height, and located in Germany. The comparison resulted in the difference at the centimeter level, which was deemed sufficient by the authors for the large scale of the survey. A recent study by Zhou et al. (2018) has also investigated a corridor shaped scene via a precalibrated metric camera sensor, and global navigation satellite system (GNSS) assisted bundle block adjustment (BBA) approach [
46]. Using their equipment, the authors will be able to achieve a 600 m corridor point cloud with an accuracy of 3.9 cm. Another study by Womble et al. (2017) performed a multiplatform remote sensing survey of one to two-story industrial structures that sustained moderate to severe damage during the 2015 tornado outbreak in Pampa, TX, USA [
47]. The collection survey was performed by deploying GBL and two UAS platforms to collect a series of oblique images from the damaged structures. This approach enabled the team to create capture affected areas, including damaged structures and related debris fields. Moreover, this data allows for a more comprehensive damage analysis to validate new wind damage prediction models and other predictive damage modeling techniques. In comparison to earlier work, the authors were also able to demonstrate the unique overhead advantage of UAS SfM point clouds in the quantification of roof damage of a school building following the 2014 Pilger, Nebraska tornado. UAS-derived point clouds can also be used for geomorphic mapping modeling, as studied by Graham (2018) [
48]. Due to the high price of lidar mapping, camera-based mapping technology has become popular, given the low-cost and efficiency. In this article, Graham (2018) compared the accuracy and resolution between SfM and lidar point clouds and discussed the idea to supplement SfM with lidar. This idea has been proposed by others, including Wood and Mohammadi (2013) [
7]. As a result, the comparison of vertical network accuracy shows both mean errors from the UAS and lidar point clouds are within 2 cm. The author also investigated hard surface precision and vegetation, which often moves in the wind. The conclusions found that SfM point clouds are sufficient for mapping purposes, except for modeling the details of vegetation. This work by Graham (2018) agrees with earlier work conducted by Fonstad et al. (2013) [
49]. Fonstad et al. (2013) investigated a UAS derived SfM point cloud with georeferenced points that can be used to create various digital elevation products of the Pedernales River in Texas (USA). The UAS SfM point cloud was compared to that of airborne lidar on the parameters of point density, horizontal and vertical precision, labor costs, expertise levels, etc. For the 200 m by 200 m irregularly shaped site of interest, the UAS survey consisted of hundreds of images, 25 ground control points (GCPs), and 15 checkpoints (CPs). While the resulting DEM has some pronounced differences, the UAS imagery was collected with a significant reduction in labor time and deployment costs. The average distance was 0.25 m, where vertical errors approached 0.60 m due to the vegetation in the area of interest. The study concluded that for their site of interest, the SfM point cloud density is lower than the lidar point cloud, but the accuracy of the SfM point cloud is controlled by the GCP collection.
Point clouds can also be used within existing infrastructure assessment workflows and inventory management. For example, the international roughness index (IRI) has been calculated to evaluate the roadway surface roughness and performance. This method assesses various longitudinal roadway profiles with the units of slope [
50]. Alhasan et al. (2015) has investigated IRI assessment using lidar and SfM 3D point clouds [
51]. In their study, five gravel and paved roadway sections were selected for testing and data acquisition with the GCPs applied. The IRI value is measured at longitudinal roadway profiles at approximately 10 cm intervals. In their work, it can be observed that corrugation started increasing while approaching intersections, which is anticipated given the acceleration and deacceleration of vehicles. The authors noted that the computed gravel roadway IRI has a range between 2 and 20 m/km; however, it is challenging to develop a statistical model of the IRI dynamic due to the limited amount of data collected. This publication demonstrates the ability to compute IRI from 3D point clouds as input, which highlights the ability to use UASs to quickly collect datasets of interest.
In another study, Zak (2016) developed a Python program to automatically compute IRI using extracted longitudinal profiles [
52]. Three test sections were selected using the proposed methodology and compared to classical onsite measurement using rod and leveling as ground truth. Prior to being input to the program, the point cloud was filtered using a moving average and extracted along the longitudinal profile. As a result, each IRI was computed from a single longitudinal profile in a 0.25 m interval. Comparing to the measurements using classical methods, the computed IRI (r = 0.94) correlates well, which proves the reliability of the developed methodology.
With the increased demand for SfM point cloud accuracy in surveying and mapping applications, research topics have started to investigate the relationship between SfM point cloud accuracy and the numbers of distributed GCPs, specifically if reliable surveying quality can be obtained and maintained using off-the-shelf consumer UAS platforms. Various case studies and GCP configurations have been evaluated in past years. For example, Tonkin and Midgley (2016) investigated a case study of a 0.145 km
2 irregular topography valley area with altitude ranges from 180 to 230 m to develop DSMs [
53]. In this case study, the numbers of GCPs vary from a minimum of 3 to a maximum of 101 in intervals of 16. The results show that the vertical RMSE ranged from 0.059 to 0.076 m for cases with four or more GCPs, while the DSM accuracy increases with the increasing number of GCPs. The DSM accuracy in this study was validated at discrete locations. Another study focusing on a 1.5 km
2 area with slopes and infrastructure was investigated by Tahar (2013) [
54]. This study included six GCP configurations with the numbers of GCP ranging from 4 to 9 for DSM accuracy analysis. According to the RMSE analysis, DSMs with 8 and 9 GCPs have the lowest error, when validated at discrete locations. As a consistent conclusion from the investigations that had a significant level of difference in the density and distribution of GCPs, the accuracy increases with an increasing number of GCPs, but the researchers were not able to indicate the limitations or best practices.
1.2. Objective and Scope
As outlined by other researchers, UAS SfM is an efficient, low-cost tool for 3D mapping and survey proposes, especially in large areas. However, accuracy is a concern and needs to be investigated, particularly at the centimeter level. Moreover, the impact of accuracy has not been demonstrated directly on roadway surfaces. Accurate UAS SfM point clouds require the deployment of GCPs to constrain the resultant data, but the quantity and distribution of GCP necessary for certain levels of accuracy have yet to be fully understood for realistic survey sites. With the investigation of GCP numbers and SfM accuracy, both lower and upper bounds are to be explored for the study. It is noted that the SfM point cloud error will vary in terms of location, quantity, and noise. This has been noted in numerous studies, including Javernick et al. (2014), where the researchers computed unevenly distributed errors varying in magnitude up to 10 cm in non-vegetative areas of the riverbank when assessing a river in New Zealand [
55].
The most closely related work by Agüera-Vega et al. (2017) investigated the UAS SfM accuracy as influenced by varying the number of GCPs [
56]. The authors collected 160 images of a 114 by 190-meter area with 72 checkerboard targets scattered uniformly for both GCPs and CPs. The number of GCPs varied between 4 and 20 for nine distinct test cases, where the lowest root mean square error was 4.7 ± 0.86 cm for the 20 GCP case. Another closely related work, Caroti and Martínez-Espejo Zaragoza (2015) investigated the San Miniato Church in Marcianella (Pisa, Italy), specifically examining small structures with ornate geometric details [
57]. The authors analyzed the front facade of this church with two variations on the number of GCPs. The authors concluded that 12 GCPs (in comparison to 6 GCPs) resulted in more accurate topographical data in comparison to lidar scans. However, the authors indicated that the error would also vary based on other parameters, including the site scale, geometry, and elevation.
While these studies and others provide some guidance on the distribution and number of GCP points, they do not quantify the errors distributed in the cloud as well as considering the potential for data overfit. Moreover, these studies concluded that increasing the number of GCPs will result in increased accuracy of the point clouds. This is in agreement with other studies [
58,
59], but no upper bound or limit is identified. Therefore, the scope of this paper is to compare the UAS SfM data to a reference lidar data on two large-scale civil infrastructure systems. Specifically, this is demonstrated on a roadway network (375,000 m
2) and a 1.0-kilometer low-volume gravel road. These datasets are significantly larger than those investigated in the previous research studies and also contain a diverse set of natural and human-made features as well as various numbers of GCPs to investigate. However, since the focus on this study is civil engineering systems, only the hardscape (e.g., pavements and gravel roadway surfaces) will be examined in detail. This study will assess the positional errors at discrete locations via checkpoints and as distributed errors throughout the point clouds using the well-cited M3C2 algorithm [
38]. Numerous cases will be explored for each site to quantify the impact of the number and distribution of ground control points to the accuracy and error with UAS SfM point clouds. The numbers of GCPs range from 0 to 14 and 36 for the two sites due to the workforce and site accessibility limitations. The comparisons are investigated in both discrete CP errors and distributed quantitative comparisons to lidar point clouds. The distributed error is essential to quantify and assess due to the unpredictable errors in SfM point clouds (e.g., [
51], which can lead to cascading impacts on decision making. Here the study includes extensive distributed error analysis using the M3C2 algorithm and its associated statistical distributions. Moreover the impact of point clouds with varying accuracies is also demonstrated. This is conducted using the IRI computation of the gravel roadway section, where the quality and the accuracy of the input point cloud is correlated to the reliability of the computed IRI. Guidance on the number and the distribution of ground control is provided.
1.3. Description of the Two Sites
The first site selected for the survey is the White River bridge, a three-span 90-m steel girder bridge, located south of Presho, South Dakota on US Route 183 (approximately at latitude of 43,704 and longitude of −100,041), with various ground coverage features in predominate cardinal directions. A paved (asphalt and concrete) highway approximately bisects the site in the north–south direction. A curvy gravel road located in the northwest portion connects to the state highway in a predominant east–west direction. In the eastern section, an active construction site is located, while the southern border is primarily representative of extensive vegetation and low-height trees along the riverbank. The survey area consists of the concrete and asphalt roadways, riverbanks, vegetation, and other surrounding regions, representing diverse and varying ground cover features. The varying elevation is also a critical component to ensure accurately surveyed elevations. The entire region of interest measures 750 m by 500 m, with an elevation range of 24 m, a paved roadway of 500 m, a gravel roadway of 450 m, and approximately 70% vegetation coverage of the entire area of interest. Due to the civil engineering-focused nature of this study, the site will be segmented to extract out only the roadway and active construction site areas for detailed numerical evaluations.
The second site selected is a 1.0-km gravel road located northeast of Waverly, Nebraska, (approximately at latitude of 40,929 and longitude of −96,502). The gravel road is located just outside of the small town and primarily supports residential traffic, with some seasonal agricultural traffic. This site is predominately linear without an intersection or curve in nature with a modest elevation range of 9 m. The site contains adjacent vegetation including corn and soybean fields, trees, etc., but the linear roadway is segmented with a width of 5 m. The corridor configuration is representative of numerous surveying and mapping sites, including various infrastructure systems of roadways, powerlines, pipelines, etc. (e.g., [
46]). Those two sites are selected due to the diverse and representative geometry features to investigate the relationship between SfM accuracy and GCP configurations, which can be further applied in more similar areas of interest for accurate analyses of civil infrastructure.