1. Introduction and Background
Structural complexity plays an important role in structuring biotic assemblages and in maintaining biodiversity of benthic marine ecosystems [
1]. The structure of intertidal and subtidal rocky reefs has been shown to be a major factor in the dynamics of resident organisms [
2,
3,
4]. Biogenic habitats provided by organisms such as oysters, mangroves, seagrass, macroalgae, and corals are critical to the integrity of their resident assemblages ([
1] and references therein). This is well studied in coral reef habitats where complexity is strongly related to important aspects of ecosystem function and service including coral cover, grazing pressure, fish density and biomass, and even tourism and shoreline protection [
5,
6,
7].
As marine ecosystems are threatened from many anthropogenic and natural forces, there has been an increasing focus on not only understanding the linkages between complexity and function but on the more basic issue of how to measure it accurately and over scales relevant to both ecological processes driving ecosystem trajectories and management [
5]. This issue is of particular concern in coral reef ecosystems where the threats to biodiversity are derived from the loss of structural complexity due to the interactive effects of stressors such as coral bleaching, overfishing, cyclones, coastal development, and outbreaks of predator species [
8,
9].
Structural complexity has been estimated over a range of spatial extents from individual coral heads to 100s of kms using a variety of tools. At very large spatial extents, variables such as bottom roughness and slope have been derived from airborne Lidar [
10,
11] and sub-surface acoustic profiling [
12,
13,
14,
15]. Over smaller areas the direct measurement of factors such as vertical relief, number of holes, surface roughness, and even fractal dimension [
5,
16,
17,
18] have been commonly estimated by divers. Among these techniques there has always been a tradeoff between area covered and resolution obtained such that very precise and accurate methodologies were often only practical over very small areas. However, recent advances in image processing algorithms and computing power has seen a proliferation of techniques to estimate structural complexity for large reef areas based on digital photogrammetry and application of structure from motion software [
16,
19,
20,
21,
22,
23].
Photogrammetry was originally used to measure the structural complexity of marine biological targets in intertidal habitats by Beck [
24]. This method soon caught on and was used to measure the structural complexity of corals and other marine organisms [
25]. Photogrammetry involves the reconstruction of a real-scale three-dimensional (3D) model of a given object or scenario from a series of overlapping photographs, taken from multiple perspectives. This method only requires a consumer-grade digital camera and basic training to ensure overlap of photos [
22]. 3D reconstruction is facilitated by an increasing number of software packages, some of them freely available and open source. Enabling non-destructive, repeatable measurements of shape, volume, and surface area, photogrammetry provides a permanent record of the 3D coral and allows it to be monitored over time, with visualisation and measurement of any change in structural complexity. The rapid speed of sampling enables the study of corals in remote reefs or deep slopes, which may have previously been intractable to study [
26].
Underwater photogrammetry has proven to be a relatively accurate technique for morphological measurements of hemispherical scleractinian corals with studies demonstrating mean differences in 3D surface areas of 0.85% [
25], <5% [
27], 13% [
26], and 1%–17% [
28] and differences in volume of 1.7% [
26], 3% [
29], to 5%–9% [
28]. Accuracy does seem to decrease for more complex branching and disc growth forms with errors in volume of 8% [
28] and 17%–24% [
29] and in surface area of 2% [
28] and 12%–78% [
27]. Comparisons of model geometry accuracy are less common although McKinnon
et al. [
30] demonstrated that photogrammetry can produce models of corals which are >90% similar to those of reference models derived from laser scans.
While studies which look at small-scale complexity (e.g., individual coral colonies), are generating photogrammetric tools to replace more conventional and destructive methods for estimating important characteristics such as surface area and volume, there is now increasing interest in applying these tools over larger spatial extents. Systems have been developed to successfully map and model large areas of benthic habitat (up to ~500 m
2) [
19,
31,
32] or even km long transects [
23] with a variety of off-the-shelf to custom-built hardware and software packages. Photogrammetry-based models derived from these various processes have been used to look at the relative benefits of various metrics of structural complexity used to characterise benthic habitats [
16,
20].
The detailed understanding of habitat complexity over a range of scales that can be delivered by photogrammetry tools has the potential to vastly improve our ability to understand, monitor, and manage benthic marine habitats. For instance, these tools could be used on coral reefs to map and monitor the effects of bleaching, cyclone damage, or even outbreaks of crown of thorns starfish on structural complexity. This motivation is driving a rapidly expanding use of techniques to estimate complexity, especially photogrammetry. However the long-term utility of these methods depends very much on the accuracy and precision with which the tools can estimate structural complexity. While the accuracy of conventional metrics like volume and surface area have been well considered at small (coral colony) spatial extents (as outlined above), there has been little work comparing either the geometry (3D structure) of derived models with those of highly accurate references (but see [
30]) or any derived metrics of structural complexity. Aside from limited work using reference models [
28] there has also been little assessment of the precision (repeatability) of photogrammetry approaches at small scales (but see [
29]). At spatial extents larger than an individual coral colony it becomes quite difficult to have any true reference model against which to compare those generated from photogrammetry approaches. Thus, accuracy is nearly impossible to assess. There is also a dearth of information on the precision of such models of benthic habitat and derived structural complexity metrics at these larger spatial extents. This gap in knowledge is especially important to address as this is the very information needed to inform questions as to the level of physical change (mm-cm?) in structural complexity that is actually possible to detect with these systems and thus their potential application for monitoring and threat assessment.
In this study we conducted an assessment of the error (precision and accuracy) involved in mapping marine benthic habitat features at two spatial extents: (1) the coral colony level and (2) the reef patch (100 m2) level. These two levels are quite relevant to the use of structural complexity data in management and ecological studies as they represent spatial extents used to ask questions about reef growth and erosion (colony scale) and relationships between structural complexity and resident assemblages (patch scale) where assemblages are typically assessed using survey areas of this size. We used readily available and thus commonly employed photogrammetric tools—hardware and software. At the colony extent level we imaged coral colonies representing six of the most common coral morphotypes on coral reefs. We quantified precision by repeatedly imaging and modeling individual colonies multiple times. For four of the morphotypes we quantified accuracy by comparing models derived from photogrammetry with those produced by high resolution laser scanning. At the patch extent we adopted a similar approach and repeatedly imaged and modeled an area of rocky reef of surface rugosity (defined below) similar to a coral reef flat. We focused our comparisons to determine the error in the 3D geometry of the models and in the derived surface rugosity metric. Both of these help specify the ability of the photogrammetry techniques used to detect change in complexity on reefs over time. We assessed the limitations of the systems used and the types of ecological questions that can and cannot be addressed with these systems. This information is critical if these novel techniques are to play a major role in either the monitoring of changes in structural complexity or simply its measurement in order to further understand its seemingly critical role as a predictor of the distribution and abundance of organisms in marine systems.
2. Data and Methodology
This study evaluated the precision and accuracy of habitat structural complexity metrics derived from high resolution photogrammetric models (mesh resolutions of 1 to 30 mm, depending on the spatial extent) of benthic marine habitats at two spatial extents; sub-meter (coral colonies) and 100s of meters (patches of benthic habitat). All tools used (hardware and software) are off-the shelf (in some cases free) components that can be used by non-experts in computer vision to suit the specific application. Precision was assessed by sampling the same coral or habitat patch multiple times to evaluate the among-sample variation (measurement error). Accuracy was assessed by comparing metrics and 3D geometry of models generated from photogrammetry with those obtained from laser-scanning (0.2 mm mesh resolution, details below). Accuracy assessment was only possible at the colony-scale. By necessity, the data acquisition and processing methodology for these two spatial scales was different as detailed below. The process flow is summarized in
Figure 1.
Figure 1.
Flow diagram of project methodology indicating the systems used to acquire images, build models and conduct precision and accuracy analysis for each of the two spatial scales, colony and reef patch. SR = surface rugosity, PG = photogrammetry and LS = laser scan. For detailed methodology of individual steps see text.
Figure 1.
Flow diagram of project methodology indicating the systems used to acquire images, build models and conduct precision and accuracy analysis for each of the two spatial scales, colony and reef patch. SR = surface rugosity, PG = photogrammetry and LS = laser scan. For detailed methodology of individual steps see text.
2.1. Subjects and Image Acquisition
2.1.1. Colony-Scale
The subjects for the small-scale assessments were individual coral colonies. In order to allow for unobstructed views of the coral without the need to physically damage them by moving them in the field, we worked with specimens of whole coral colonies representing the major morphotypes loaned from the Australian Museum. Up to 10 individual coral colonies ranging in size from 15–50 cm diameter of six different morphologies were used in the small-scale study; (1) small-sized massive (~12 cm diameter); (2) medium-sized massive (with crevices, ~30 cm diameter); (3) tabulate; (4) coarse branching; (5) foliose; and (6) bottlebrush. As the thin layer of tissue on living corals only contributes a small percentage to their complexity [
33], use of the coral skeleton allowed us to document biologically relevant levels of complexity.
Image acquisition for each coral colony was conducted by snorkel in a pool at a depth of 1.2 m, which offered similar conditions to those of a shallow coral reef lagoon for underwater photogrammetry, with natural light variation (
i.e., light rays filtering and reflecting on bottom), passing swimmers in the near background (
i.e., passing fish), and clear water (although sea water can be slightly more turbid than fresh water;
Figure 2). Images were taken using a 24 MP Sony NEX-7 digital camera, with a fixed Sony 16 mm f/2.8 E-mount wide-angle lens to reduce spherical distortion. The camera was in a waterproof Nauticam housing with a dome port. A dome port was chosen over a flat port in order to eliminate refraction and loss of image quality along the edges of the frame.
Figure 2.
Examples of four of colonies used in this study showing a captured image (left panel) and the resulting fully textured model (right panel); (a) Bottlebrush; (b) Massive-medium (~30 cm diameter); (c) Massive-small (~12 cm diameter) and (d) Tabulate.
Figure 2.
Examples of four of colonies used in this study showing a captured image (left panel) and the resulting fully textured model (right panel); (a) Bottlebrush; (b) Massive-medium (~30 cm diameter); (c) Massive-small (~12 cm diameter) and (d) Tabulate.
A pilot study established that ~120 images was a suitable number to cover all perspectives required and enable reconstruction of a representative 3D model across a range of simple to complex coral morphologies. The image acquisition process involved placing the coral colony on the floor of the pool in a fixed position, similar to its natural orientation when attached to reef substratum, with a scale bar. Then, ~120 overlapping photographs at a rate of ~1 photo every 1–2 s with no flash were taken for each coral from different viewpoints using a standardised technique. The technique involved doing two 360° passes around the colony (~17 images per pass) and repeating this for distances of 1 m, 60 cm, and 30 cm away from the colony. Then a final two passes were done, in arcs directly over the top of the colony with the second being perpendicular to the first (~15 images per pass). Image acquisition took ~5–7 minutes per coral.
2.1.2. Patch-Scale
The focus for this work was on mapping larger areas of benthic habitat. For this study we used temperate rocky reefs as a focal system for the trials. An area of rocky subtidal reef at Shelly Beach in Sydney NSW, Australia was imaged using a custom-designed system. The habitat was at about 3–4 m depth and had a slope of about 45°. The mapping system consisted of an aluminium frame, 2 m long by 30 cm deep, to which three cameras were attached facing downward and equidistant apart (1 m between cameras). Cameras were GoPro Hero4 Black cameras (fixed F2.8 aperture, approx. 22 mm equivalent field of view, 1/2.3 inch sensor with 4000 × 3000 pixels) which were fitted with manufacturer underwater housing with a flat port.
Mapping was completed by first laying three to four visual references and two scale bars of known length (50 cm pvc cylinders marked at 10 cm intervals) over a ~20 m area of reef, and slowly (20 m/min) swimming the mapping rig at 2–3 m elevation over the bottom. The cameras were set to capture an image every second (wide field of view, 12 megapixel resolution). This protocol resulted in an image footprint size of approximately 5 m × 4 m and 80% overlap in imagery in the direction of travel. The whole system (all three cameras) covered a swath of approximately 7 m. Three passes were conducted in parallel to the transect line with the camera nearest to the previous pass having at least 50% overlap in image footprint with the nearest camera from the previous pass. Following this protocol, an area of approximately 200 m2 of reef was covered in approximately 10 min, generating 800–1000 images from the three cameras.
2.2. Model Building
The processing systems for colony and patch scale studies differed slightly but in both cases the 3D textured models were created from acquired images using a five step process: (1) aligning photos by feature detection and matching; (2) building of a sparse point cloud; (3) building of a dense point cloud; (4) mesh construction and (5) photo-texturing. This process was completed on Windows 7 workstations (minimum specifications of 64 bit OS, 64GB RAM, 2GB GPU, and quad-core 3.0 GHz CPU). Different processing systems were used for the patch and colony extent modeling as the purpose of this study was to evaluate precision and accuracy at each spatial extent using the types of tools currently being used and thus viewed as most appropriate for these different tasks. Each process is described below.
2.2.1. Colony-Scale
It is not uncommon for photogrammetry of features of the size of coral colonies to be undertaken using simple imaging systems (SLR or even point and shoot cameras) and small numbers of photos (10s–100s maximum) and so are amenable to modeling using free, open source software. We constructed models of the colonies using the free, open source software packages VisualSFM to build point clouds and MeshLab to generate meshes and do photo texturing. Feature detection in VisualSFM used a scale invariant feature transform (SIFT) algorithm [
34] and generated a network of camera poses from which the sparse 3D point cloud of the model surface was generated. The sparse point cloud output is then processed by the Clustering Views for Multi-view Stereo (CMVS) package, which produces a much denser point cloud, which retains the colour information from the original image pixels. This package includes Patch-based Multi-view Stereo version 2 (PMVS2) which preserves only static structures in the point cloud, so any unwanted moving objects (e.g., fins, fish) that may have been photographed are excluded. PMVS2 is also impervious to any differences in image colour caused by exposure settings, lighting conditions, or white balance [
35].
The dense point cloud was then imported into Meshlab where a polygon mesh was constructed using the Poisson Surface Reconstruction algorithm [
36]. The scale bar in the image was used to scale the model into real units and then the scaled model was clipped to exclude anything other than the colony itself (such as the floor around the colony) and touched up by filling holes if needed. Finally, the photo-texture was projected onto the mesh using colour information from the original images (
Figure 2, right panels). The final models had average (±SD) mesh resolutions (distance between neighbouring vertices) ranging from 1.2 mm (±0.40) to 2.8 mm (±0.57).
2.2.2. Patch-Scale
In contrast to the colony scale models, patch-scale modeling required more robust software solutions capable of processing 1000s of images and the automated elimination of non-static features in those images (mobile marine life and especially moving macroalgal habitat). Patch level models were reconstructed using the software package Photoscan Professional v1.1.6 (AgiSoft LLC). Photoscan uses a similar workflow as VisualSFM to build 3D models based on overlapping photos of the same areas from different angles and has been used in other studies which have modeled larger areas of benthic marine habitat as done here [
16,
19]. This process is highly dependent on the quality of photos, overlap between photos, and capturing all surfaces from multiple angles. First, photos are aligned by identifying common, invariant points, which results in a sparse point cloud in 3D space. This allows the software to calculate the location and orientation of cameras that took them. Second, alignment is optimized by the program based on camera and lense properties specified in image metadata. GoPro cameras set to wide produce barrel-distorted images. The Photoscan software accounts for this distortion with the “fit k4” alignment option in addition to the inbuilt lense distortion algorithms. Third, a dense cloud of points is generated by populating the sparse point cloud based on comon points in the photos and camera locations. Fourth, a mesh of three-sided polygons is built over the dense cloud to create a continuous 3D surface. Finally, a fitted texture is created by draping the original photos over the mesh. Specific settings are given in
Table 1.
Table 1.
Settings used for processing of patch-level models in Photoscan Professional (v1.1.6; Agisoft LLC).
Table 1.
Settings used for processing of patch-level models in Photoscan Professional (v1.1.6; Agisoft LLC).
Process | Parameter Choices |
---|
Align photos | accuracy low, pair preselection disabled, key point limit 40,000, tie point limit 1000, do not constrain features by mask |
Optimize alignment | all properties yes, including fit k4 |
Build dense cloud | medium quality, mild depth filtering, do not reuse depth maps |
Build mesh | arbritray surface type, source data-dense cloud face, count medium, interpolation enabled, all point classes |
Build texture | generic mapping mode, texture from all cameras, mosaic blending mode, texture size 4096, texture count 1, no color correction |
The resulting models were clipped to exclude bordering areas of incomplete coverage and cleaned for obvious defects (e.g., detached components, holes, any bowl effect around edges). This process resulted in 3 meshs with an average (±SD) resolution of 30.3 (±2.82) mm each of which covered an area of approximately 200 m2. Model building took approximately 3 h of manual work plus 10 h of automated processing from photo collection to final mesh. This time was broken down into the following tasks (times approximate): 20 min of underwater photography (setup and mapping), 60 min of photo download and archiving, 30 min of model set-up, 10 h of automated model building, and 60 min of mesh checking, scaling, and cleaning.
2.3. Metric Derivation
For each model (coral colony or region of benthic habitat) we estimated the ratio of the 3D surface area to the 2D planar area (from the plane normal to the model), hereafter referred to as surface rugosity (SR). Surface rugosity is a metric similar to linear rugosity [
37], but which accounts for the three-dimensional structural complexity of the coral [
20]. This metric of complexity is commonly used in benthic remote sensing applications to assess bottom complexity, though typically over much larger spatial extents and at much lower resolutions [
12].
2.4. Analyses of Precision and Accuracy
2.4.1. Colony-Scale
The precision with which coral colonies were modelled was assessed by repeatedly imaging and modelling one colony of each of the six morphologies 7 to 10 times (depending on morphology). We then derived SR estimates from each model and calculated their average, standard deviation (SD) and coefficient of variation (CV = 100 × average/SD) per morphology.
We also assessed differences in the geometry of 3D models using the “deviation analyses” tool in Geomagic Studio (3D Systems). The routine first aligned all models (“best fit algorithm”) for a morphology using best fit alignment and compared all models computing the average and SD of all distances between all pairs of models for all spatially matched points on the surface of each model. We evaluated the spatial distribution of these errors using heatmaps of these SD values overlaid on the average of all models per morph (using the “create average model” tool in Geomagic Studio).
Accuracy at the colony scale was assessed by comparing the average model from the repeated precision trials with that derived from laser scanning of the colony. Laser scanning was conducted by WYSIWYG 3D ® of Sydney Australia using a Nikon MMDx100 laser head, mounted on a Faro Platinum arm, with a combined system volumetric accuracy of ±0.05 mm. Real-time point cloud rendering captured the model at 0.2 mm resolution using Geomagic Studio point cloud software.
We compared the SR value from each photogrammetric model to that derived from the laser scan. From this we generated the average, SD and CV of the differences per morphology. We also used a Z-test (normal deviate) to test the hypothesis that the SR value from the laser scan model was not significantly different from that of all the models derived from photogrammetry.
The accuracy of model geometry was assessed by comparing the average model for each morphology with the laser scan using the “deviation analyses” tool in Geomagic Studio. The average model was created by aligning (“best-fit alignment”) all models for each morph and using Geomagic Studio’s “create average tool”. Accuracy was assessed as the average of the absolute value of all distances between all pairs of spatially matched points on the surface of each of the two models.
2.4.2. Patch-Scale
The precision of the patch-scale modelling technique was assessed by repeatedly mapping the exact same area of habitat three times over a period of one hour on the same day. Once rendered and cleaned (as described above), all models were imported into Geomagic Studio and precisely aligned using anchor points. The three models were then clipped to an area of about 120 m2 which was common to all three models. Model geometry was compared in Geomagic studio in the same manner outlined above for the colony scale. This process generated the average and SD of all distances between all pairs of models for all spatially matched points on the surface of each model.
The precision of SR values derived from the models was assessed by uniformly sub sampling each mesh with a series of non-overlapping virtual quadrats in order to generate spatially matched (exact same area on all meshes) estimates of SR for each of these quadrats. This was done by first exporting each spatially aligned and clipped mesh from Geomagic in VRML (Virtual Reality Modeling Language) format, exporting the faces to X3D format using MeshLab (
http://meshlab.sourceforge.net) and then conducting the spatial sub setting routine using a custom script in the Scala programming language (
https://github.com/shawes/Mesh3D). The script first identified the largest maximum rectangle that fit within the bounding box of all three passes. Secondly, using coordinate geometry, this rectangle was sub-divided into approximately square (width and length provided as input), equal sized virtual quadrats. We then used a ray-casting algorithm to identify all faces whose centroid was inside a given virtual quadrat.
The 3D surface area of the mesh within each virtual quadrat was then calculated by summing the areas of all faces of the mesh that fell inside the quadrat. The surface rugosity within each virtual quadrat was then determined by dividing the 3D area by the 2D area of the same portion of the mesh. Because the mesh is made of triangular faces and the centroid was used to determine quadrat membership, the 2D projection of this mesh will not exactly fit within its nominally bounding virtual quadrat. The similarity of the 2D area of the virtual quadrat to that of the actual piece of mesh will decrease as the virtual quadrat size becomes closer to the resolution (internode distance) of the mesh. To avoid this dependence, we use the most accurate estimate of the 2D area of the piece of mesh of interest which is simply the sum of area of the 2D projection of all faces in a virtual quadrat onto the common 2D plane for the model. Note that there is no real-world data on the slope of the habitat in the model and as such, the 2D planar area of each quadrat is essentially based on a common plane-of-best-fit for the entire region. The approach of using a plane-of-best-fit when calculating surface roughness estimates such as this is generally favoured over the use of the true, real-world horizontal plane. This is because using plane-of-best-fit de-couples slope and SR, ensuring that change in elevation does not influence surface roughness calculations [
20,
38]. This approach was completed for five different nominal sizes of virtual quadrats, 0.25 m
2, 1 m
2, 2.25 m
2, 4 m
2, and 9 m
2, giving sample sizes (number of non-overlapping quadrat which could be fit on the surface) of 456, 114, 48, 27, and 12 respectively.