Missing the Forest and the Trees: Utility, Limits and Caveats for Drone Imaging of Coastal Marine Ecosystems

Abstract

Coastal marine ecosystems are under stress, yet actionable information about the cumulative effects of human impacts has eluded ecologists. Habitat-forming seaweeds in temperate regions provide myriad irreplaceable ecosystem services, but they are increasingly at risk of local and regional extinction from extreme climatic events and the cumulative impacts of land-use change and extractive activities. Informing appropriate management strategies to reduce the impacts of stressors requires comprehensive knowledge of species diversity, abundance and distributions. Remote sensing undoubtedly provides answers, but collecting imagery at appropriate resolution and spatial extent, and then accurately and precisely validating these datasets is not straightforward. Comprehensive and long-running monitoring of rocky reefs exist globally but are often limited to a small subset of reef platforms readily accessible to in-situ studies. Key vulnerable habitat-forming seaweeds are often not well-assessed by traditional in-situ methods, nor are they well-captured by passive remote sensing by satellites. Here we describe the utility of drone-based methods for monitoring and detecting key rocky intertidal habitat types, the limitations and caveats of these methods, and suggest a standardised workflow for achieving consistent results that will fulfil the needs of managers for conservation efforts.

1. Introduction

Biogenic habitats provide key services to ecosystems. They constitute the most basic proxy for understanding the relative health of ecosystems. Increasingly, these biogenic habitats and the ecosystems they support are threatened by extreme climatic events [1,2,3] and a changing nearshore environment [1,4]. Worldwide, we are grappling with the cumulative consequences of human activities. Maintaining ecological resilience in the face of global climatic changes and extreme events inevitably involves managing the stressors that can be controlled at local or regional levels. To understand the spatial and temporal extent of interventions, however, we must have suitable ecological detection and monitoring at the appropriate spatio-temporal scales.

Kelp and fucoid algae (Phaeophyceae) are present from tropical to polar oceans [5] and some genera have truly global distributions [6,7]. These large photosynthetic organisms support a large biomass, high productivity, and contribute globally to the fixation and sequestration of carbon [8,9]. For example, southern bull kelp (Durvillaea spp.) can reach 8–10 m in length and is an ecosystem dominant across much of the temperate southern hemisphere and subantarctic islands [10]. It provides habitat, alters physical properties of local environments, defines ecosystem structure [11], and is highly productive [12]. It also occurs primarily on the most wave-exposed rocky reef habitats. Not only has this species been greatly reduced by marine heat waves that caused additional localised extinctions [1], but large-scale coastal uplift from a 7.8 magnitude earthquake [13,14,15] severely affected populations, causing complete loss across uplifted rocky reef habitats [16]. These impacts occurred over 130 km of a sparsely inhabited, very heterogenous coastline featuring platforms, rocky outcrops and boulder fields that made in-situ sampling difficult or impossible in many places. Identifying remnant populations of this greatly impacted habitat-forming species therefore required new tools, first to assess the extents of loss and then to establish routine practices that could accurately track recovery dynamics or aid in the implementation of intervention measures such as restorative practices [17].

Aerial imagery for surveying has been used for decades, and earth-observation satellites are now a staple of ecological monitoring globally [18,19,20,21,22]. There are, however, several challenges to using these techniques, particularly low-resolution satellite images, for rocky reef ecosystems [23]. Perhaps first and foremost is the very highly localised and patchy diversity evident in nearshore marine systems [24]. In the focal area, over 200 species have been recorded, and there is an average of around 20 species per square meter in the lower intertidal zone [16]. The varying sizes of individuals of each species, ranging from sub-cm to m, and the layering of their canopies [25] render them difficult or impossible to see in their entirety from above. Because individuals are usually difficult to distinguish even with fine-scale in-situ surveys, the percentage cover of attached algae and mat-forming invertebrates (such as mussels) is usually the metric of choice. These ecosystems can be assessed adequately from above only on very low tides when swell conditions are suitable, which must also coincide with a favourable sun angle. Finally, given all of the complexities of sampling and of the communities themselves, validation involves a great deal of in situ work to identify exactly what is seen in images.

Unlike satellites, unmanned aerial vehicles (UAVs or drones) can overcome many of these challenges. They have very high pixel resolution, deployment can be timed to coincide with appropriate meteorological and oceanographic conditions, and in-situ validation sampling can be completed simultaneously [23,26,27,28,29]). Accurately relating remotely measured spectral signatures to in situ observations provides the key mechanism for building effective detection algorithms and assessing their accuracy [27]. Spectral libraries provide promise for remote taxonomic surveys [30], especially when aligned with low elevation hyperspectral imaging surveys [29,31], but challenges remain to deal appropriately with fundamental and realised spectra of species across seasons and under multiple scenarios of water coverage [29]. Appropriate validation of observations is also an area with no widely accepted protocols, although embedding transects has been widely used [23,26]. Previous work on UAVs and rocky reef imaging [23,26,27,28,29] show the value of small-scale aerial imagery for marine rocky reefs, and the difficulties of using passive remote sensing (Figure 1). While satellite imaging sensors are increasingly offering sub-metre pixel resolution (e.g., Maxar, 15–30 cm resolution), the high return frequency satellites (e.g., Sentinel-2 and Landsat-8) offer a minimum pixel size of 10–30 m, allowing only broad detection of emersed vegetation visible in near-infra-red (Figure 1).

This paper outlines the utility, limitations and caveats to the use of drones in monitoring vulnerable marine rocky reef habitats and proposes a consistent standardised approach to the capture of imagery and validation samples to ensure that such imagery meets the requirements of agencies tasked with management and conservation of vulnerable or threatened species. We use two examples to illustrate the utility of drones for conservation management: (1) The 2016 Kaikōura earthquake, which shifted relative sea levels, caused mass mortality of key habitat-forming macroalgae (especially of the southern bull kelp species), and shifted assemblages to ephemeral red and green algae [16,32]; and (2) highly wave-exposed rocky reef platforms located on the West Coast of New Zealand.

2. Materials and Methods

Both RGB and multispectral imaging were used from unmanned aerial vehicle (UAV) platforms to distinguish between multiple species of macroalgae and non-biological substrata. One major aim of using these aerial platforms was to identify remnant populations of southern bull kelp Durvillaea poha/antarctica (generally indistinguishable species in the field, but most likely D. antarctica) which was dramatically affected by the Mw7.8 Kaikoura earthquake in November 2016. The aerial drones were also used to assess the remaining coverage of large brown algae, once dominant along the entire length of this coastline, and the transition to ephemeral green and red algae and increasing coverage of bare rock [16]. An additional impact was that remnant populations of Durvillaea, in particular, also suffered high mortality after the marine heat wave and high air temperatures that affected southeast New Zealand in the summer of 2017–2018 [1].

Flights were generally completed during the lowest tide series of the month and were completed within 30–60 min each side of peak low tide (tidal amplitude of this region ranges between 1.2 and 1.8 m). Marine conditions were typically limited to wave-heights less than 1.0 m and ideally coincided with light offshore wind conditions. This combination of conditions also needed to coincide with favourable sun angles (middle portion of the day), resulting in a relatively specific set of operational parameters that directed the flight planning stage.

2.1. RGB Imaging of Rocky Reef Ecosystems

An imagery acquisition campaign was implemented for 8 reefs that experienced different degrees of uplift during the Kaikōura earthquake. Imagery was collected by a DJI Mavic Pro 2 drone (SZ DJI Technology Co., Ltd., Shenzhen, China) with a Hasselblad 20 mm fixed focal length lens, and 1” CMOS sensor capturing 20-megapixel files. Flights were at an elevation of 30 m and images taken with an overlap of 85% and sidelap of 80% to achieve a ground sampling distance (GSD) of 0.7 cm.

The 8 sites were spaced across 130 km of coast and represented a gradient of vertical deformation from the earthquake (

Table 1. Details of sites for which aerial imagery was captured along the Kaikōura coast affected by the November 2016 7.8 Mw earthquake, including the Latitude and Longitude, and vertical deformation (m) [32].

Site Name	Eco-Region	Coordinates	Earthquake Vertical Deformation (m)
Cape Campbell	North	−41.724430, 174.2772	1.1
Gate Reef	North	−41.812753, 174.216667	1.4
Waipapa Reef	North	−42.209011, 173.879163	5.5
Okiwi Reef	North	−42.218046, 173.87096	2.3
Paia Point	South	−42.472953, 173.536960	1.1
Omihi Reef	South	−42.49100, 173.523796	1.7
Oaro North	South	−42.516454, 173.508488	0.0
Oaro South	South	−42.521991, 173.506124	−0.3

). To analyze RGB imagery, the combined orthoimagery was stitched together using Agisoft Metashape™ (Agisoft LLC, St Petersburg, Russia) and segmentation procedures were performed using Arc GIS Pro™. The segmented imagery (3-band RGB) was classified using support vector machines (SVM) trained using 50 training samples per class (Arc GIS Pro™, ESRI, West Redlands, CA, USA). Samples were polygons selected around habitat classes encompassing, in some cases, several hundred pixels. Determination of sample classes were established based on the species and taxonomic categories identified from in situ surveys. Each location was separated into 5 clips of 100 × 100 m and assessed for relative coverage of several key habitat types, including bull kelp, other brown algae, green algae, red algae, coralline algae, submerged algae, water, and bare rock.

2.2. Multispectral Imaging

Multispectral imagery was captured at an exposed and poorly accessible rocky reef habitat on New Zealand’s West Coast (−42.090285, 171.337623). A DJI Matrice 600 (SZ DJI Technology Co., Ltd., Shenzhen, China) equipped with both an Airphen^® (Hyphen, Avignon, France) six-band multispectral camera and a Sony mirrorless RGB camera was used to capture imagery. The Airphen^® multispectral camera has a focal length of 8 mm, a sensor resolution of 1280 × 960 pixels, and six synchronized global shutter sensors, centered at 530, 570, 630, 670, 710, and 750 nm (band width 10 nm). The three bands of the Sony^® (Konan, Minato-ku, Tokyo, Japan) RGB camera span the visible wavelengths (400–700 nm), with large overlap between blue and green and green and red, but little overlap between red and blue. The Sony camera was an A5100 with a 15 mm Voitlander^® (Braunschweig, Germany) rectilinear lens, providing final images of 6000 × 4000 pixels.

To georeference the RGB and multispectral imagery, c.16 ground control targets were laid out evenly across the Region of Interest (ROI). These targets were 0.5 × 0.5 m with high contrast blue and yellow squares to identify accurately the center of the target. Each target was surveyed using a high accuracy Real-Time Kinetic Global Positioning System (RTK-GPS; Trimble Inc., Sunnyvale, California, United States). Furthermore, between 200–400 surveyed “validation points” of up to 12 species (Features of Interest, FOI) were collected across the ROI using the same RTK-GPS equipment. These in situ point surveys allowed the collection of large numbers of rapid samples of various habitat types. Tying these “validation points” to the aerial imagery was achieved through accurate tying of “targets” to the measured coordinates during both the image stitching phase and at the whole-ortho scale. The resulting dataset provided a foundation for accuracy assessments of habitat classification and provided even coverage of multiple habitat-types compared to embedding transects within aerial imagery (see [23]).

2.3. Analysis and Validation

The combined RGB and multispectral imagery captured from the DJI Matrice 600 was used to produce multiple products. RGB imagery (8 bit) was stitched together using Agisoft Metashape™ (Agisoft LLC, St Petersburg, Russia). The final orthoproduct was a three-band image with a final pixel size of 1.25 cm² per pixel. Multispectral images (32 bit) were stitched together using Agisoft Metashape™, as well as the Airphen™ plugin for multiband imagery. The final products included a six-band orthomosaic with 2.5 cm² pixels, a 3-band RGB orthomosaic, and a Digital Elevation Model (DEM). Reflectance was calibrated using a reference panel and the radiometric calibration tool in Agisoft Metashape™ and the Airphen™ plugin. Furthermore, a single-band Normalised Difference Vegetation Index (NDVI) orthoimage was calculated using the red, and near-infra-red (NIR) bands of the multispectral camera (Equation (1)).

NDVI equation:

$$\mathrm{NDVI} =\frac{\mathrm{NIR} - \mathrm{Red}}{\mathrm{NIR} + \mathrm{Red}}$$

(1)

Habitat classification procedures were done on a per-pixel basis for multispectral imagery (6-bands + NDVI) using support vector machines (SVM) trained using 50 training samples per class (Arc GIS Pro™, ESRI, West Redlands, CA, USA). Unlike previous examples [23], RGB and multispectral image products were not composited due to alignment challenges over complex 3D habitats. Instead, RGB imagery was used to identify training samples within the 6-band multispectral image. Furthermore, the RGB image provided an additional quality control layer to ensure alignment between validation samples, and the species/habitats they represented.

Training samples were collected as polygons selected around habitat classes encompassing, in some cases, several hundred pixels. Determination of sample classes was established based on the species and taxonomic categories identified from in situ surveys. The in-situ validation samples were retained for accuracy assessment to avoid overfitting. The assigned classes were: Durvillaea spp.; other brown algae (mostly Carpophyllum maschalocarpum, but also including the occasional Cystophora spp.); Ulva spp. (green, sea lettuce); red algae; coralline algae (including articulated and crustose coralline algae); a generic algal class not readily identifiable to species due to increasing water depth (but still identifiable as vegetation); bare rock; water (with no visible submerged vegetation); and shadow. User and producer accuracy were assessed by an equalised stratified random sampling procedure. Cohen’s Kappa of the combined agreement between the classified dataset and the validation sample was also computed. Our final workflow was split into three general steps: flight plan development, in situ image and data collection, and data processing and analysis (Figure 2).

Multivariate species data were analysed by averaging the coverage of each algal taxon over the 5 plots from replicate clips (100 m² area). Comparisons of the large brown algal species were done on the square root of density using pairwise Bray–Curtis resemblance matrices. These matrices identify the similarity of multivariate datasets to describe the similarity between samples. Comparisons of macroalgal composition were done on the square root of cover (%) using pairwise Bray-Curtis resemblance matrices (using the statistical package PRIMER 7 ©, Plymouth Routines in Multivariate Ecology Research). Variation in the composition of functional groups between sites (variably affected by earthquake uplift) was done using PERMANOVA (permutational analysis of variance). Community composition data were explored using multidimensional scaling plots of similarities in the combined composition of key habitat types to show differences between eco-regions and uplift values (Table 1).

3. Results

3.1. RGB Imaging of Rocky Reef Ecosystems

Using UAV-based RGB imagery across earthquake-affected sites spanning 130 km of the Kaikoura Coast, we assessed the relative cover of several key habitat types (Figure 3). General trends in key habitat types showed a high cover of bare rock at uplifted sites, as well as a large cover of green and red algal groups, especially at northern sites (Figure 3). Bull kelp covered up to 20% of southern rocky reef and 8% of northern rocky reef (within 100 m² regions), with declining cover observed at increasingly uplifted reefs. This was in stark contrast to the pre-earthquake composition of these ecosystems where consistent coverage of bull kelp and brown algae were observed (see [16]). Coralline algae was also notably affected by coastal uplift.

Drone imagery made it possible to see and assess remnant populations of Durvillaea attached to offshore rocky outcrops that were inaccessible by land and sea (Figure 4). For example, at Cape Campbell (Figure 4) there was only marginal coverage of bull kelp evident at land-based sampling sites (such as the validation sites used in this study), but from aerial images there was between 2–8% cover among the randomly sampled sub-regions (five 100 m² clips). The region accessible to shore-based researchers had few examples of Durvillaea (large, brown-bladed alga; Figure 4), but offshore rocks had become a refuge for these populations that were greatly affected by air and marine heat-waves, and earthquake uplift.

Multivariate analysis was done of the composition of 7 habitat types, which included the physical and biogenic features of bare rock, the green Ulva spp., Durvillaea, other brown algae, red algae, coralline algae, and submerged vegetation. The analysis showed that the degree of uplift affected the relative composition of habitat types and that the “Northern” and “Southern” sites had inherent differences (Figure 5). Permutational ANOVA showed a significant effect of uplift on habitat cover composition (Psuedo-F_6,39 = 21, p < 0.001). These differences were primarily driven by red and green algae in the north, and bull kelp and brown algae in the south.

3.2. Multispectral Imaging

Method refinement at additional sites clearly showed the utility of multispectral imaging for detecting populations of bull kelp, and also highlighted the value of RGB imagery for better enabling manual investigation of images (Figure 6). Furthermore, the safe limits of access for shore-based researchers only approached the edge of the distribution of bull kelp (Figure 6A), whereas aerial images indicate populations in the low shore (Figure 6B). Incorporation of DEM layers (Figure 6C) enabled more refinement of the ROI and incorporation of natural zonation patterns into classification routines. Likewise, the presence of red-edge and NIR bands (Figure 6D) and the use of band arithmetic procedures (e.g., NDVI; Figure 6E) allowed the relative coverage of photosynthetic and heterotrophic habitat-formers to be assessed (e.g., algal beds and mussel beds). The final classified image (Figure 6F) showed very high agreement with the in-situ samples (

Table 2. Producer and user accuracy of support vector machine habitat classifications as compared to in situ validation samples.

Species/Habitat	Producer Accuracy	User Accuracy
Ulva	0.94	0.63
Brown algae	0.97	0.92
Red algae	0.99	0.93
Coralline algae	0.998	0.91
Durvillaea	0.99	0.98
Mussel beds	0.994	0.89
Rock	0.994	0.86
Water	0.998	0.98
Seafoam	1	0.996
Total	0.986	0.90

), with overall accuracy of 0.94 (Cohen’s Kappa). Only a single class (the green Ulva species) had agreement lower than 85% for user accuracy.

4. Discussion

We found that over accessible reefs there was remarkable concordance between results from drone images and those from in situ surveys. It was clear from immediately after the earthquake that the damage to marine habitats depended considerably on the degree of coastal uplift. The major taxa underpinning biogenic habitats were the large brown algae, which provided most of the three-dimensional structure supporting other species [16,33,34]. Our drone surveys showed that both the largest species (the bull kelp Durvillaea) and a suite of smaller brown algae (particularly Carpophyllum maschalocarpum and Cystophora spp.) generally declined in cover with increasing reef uplift. It must be remembered that these analyses were done around two years after the earthquake, so they represent lasting effects and relatively poor recovery. As large brown algae decreased in sites, bare rock increased, and much of this was taken over by fleshy red algae and green sea lettuce (Ulva spp.). These results are entirely in line with those of the detailed monitoring done laboriously by field teams, who spend hundreds of hours annually in the field doing surveys [32]. However, the relative detection of the vulnerable Durvillaea poha/antarctica species [1] revealed that offshore refuge populations were more frequently detected and quantifiable by aerial drones than by field researchers. Given that accessibility frequently dictates which rocky reef habitats are sampled by field researchers, aerial drone platforms can fill the gaps of field sampling campaigns and ensure that these marine forests are not missed, and remnant populations are delimited to enable appropriate intervention measures.

Several caveats and limitations to the use of drones for such surveys were obvious from this study. First and foremost, any form of diversity monitoring for recovery dynamics or conservation must have clear goals and objectives. Without these, data are not clearly tethered to the purpose of their capture. Spatial limitations, repeatability, and temporal sequences in often seasonally abundant species all affect how and to what degree drones will be useful. The key limitations of UAVs for monitoring and conservation objectives are:

• Detection of unique spectral signatures is greatly impeded by overlying water, leading to further aggregation of unique habitat/species classes.
• Features of interest (FOIs) smaller than c. 5–20 cm² require very low elevation flights to define individual FOIs by >1–4 pixels. This greatly limits:

  ▪

  The use of imagery for collection of “training samples”.

  ▪

  The size of the area that can be covered under the limitations of UAV batteries, and tidal cycles.

• Full taxonomic inventory is not readily achievable (even for macroalgae) from aerial imagery alone.

Although the combined use of hyperspectral imaging and species-specific spectral libraries [29] have great potential to increase taxonomic resolution of aerial surveys, the complex optical environment limits the practical separation of multiple species. This is further exacerbated by the fact that most intertidal species exist within tidal zones because of the shelter, moisture, and micro-habitats provided by large species that protect them in their understory at low tide [35,36]. These species are at best only partially and occasionally visible from above. Drone imaging therefore misses most of the trees—the numerous individual small species that comprise much of the biodiversity of reefs. As is evident in quadrat-based field surveys of intertidal algal communities dominated by seaweeds, there would be typically only a few species visible from above [35,36].

While we must be mindful of limitations of these tools, there is clear utility for monitoring and mapping key habitat-forming species [29], including in a conservation context where species are greatly limited in their spatial distribution. It was clear that with adequate ground-truthing we achieved an accurate and precise estimate of algal by major taxa and bare rock cover, both of which are critical to understanding legacy effects and recovery. Accuracy assessments showed that RGB and multispectral datasets provided accuracies of c. 80%, while 9-band composite imagery exceeded the often-cited 85% threshold of agreement [37], with c. 89% agreement between supervised classification and in situ validation samples [23]. Here we show that >90% accuracy can be achieved with 6-band multispectral imagery to detect a mixture of invertebrate species (i.e., mussel beds) and algal species, including the vulnerable and ecologically important southern bull kelp. Importantly, drone use enabled sampling of the full range of habitats, which is difficult or impossible to sample by field teams given the heterogeneity of habitats and difficulties of traversing the terrain along such a coastline. Once ground-truthed and calibrated, this is a transferable technology that can save hundreds of person-hours of laborious field work, obviate the continual need for identification and field specialists for long periods, and thereby give a much more spatially complete indication of recovery of the coastline. The utility of drone surveys can only increase as ground-truthing (e.g., spectral libraries, [30,38,39,40]) improves for more species under different circumstances and different types of disturbances.

We show that the application of UAVs is well-suited to monitoring and detecting the ecologically important and highly vulnerable southern bull kelp species, as well as similar species of canopy-forming macroalgae [23,24,25,26,29]. Although it can be difficult to retrieve cloud-free satellite imagery of coastal locations at low tide and under calm oceanic conditions (e.g., Figure 1), drone imagery can be applied strategically to produce high resolution moderate-scale maps that can be used to delineate canopy-forming macroalgae, patches of benthic invertebrates (such as mussels), and impacted areas with little or no vegetation cover. Availability of high-resolution satellite imagery (e.g., Maxar, 15–30 cm pixels) will increasingly enable more passive methods for understanding these ecosystems, but limitations in tasking, costs, and spectral resolution remain.

The key to robust use of drone-based maps is the collection and alignment of validation samples. Following initial methodology for embedding transects into aerial imagery (e.g., Tait et al. 2019 [23]), we propose that validation samples are collected with a high accuracy survey system. This enables a few key benefits over embedded transects: (1) There is no threat of transects being moved by waves; (2) habitats not observed within transects can be sampled; (3) habitat classes or species can be sampled evenly for robust training and validation. We endorse the use of “targets” or GCPs visible in the aerial imagery that are georeferenced using high accuracy real-time kinetic global positioning systems (RTK-GPS). Tying the visible GCPs to the habitat samples (also sampled with RTK-GPS) provides a robust mechanism to position samples accurately in relation to the GCPs and limit the mismatch of habitat samples from the imagery. The use of precision RTK-GPS also provides accurate placement of the entire orthoimage for repeat sampling and enables robust calibration of shore height, which provides important information on the relative zonation of biogenic habitats and can be used to mask habitat outside of the area of interest. Our proposed workflow for the planning of flights, completion of in-situ sampling and analysis of data will enable robust classification and accuracy assessments of key biogenic habitats on rocky reef platforms and aid in the detection or delineation of rare or vulnerable species.

5. Conclusions

As spectral libraries for coastal algal species accumulate [30,40,41,42], and as the use of finer spectral resolution imaging systems become common practice, the generation of classified products will increasingly yield greater taxonomic richness and require less validation [29]. Given the length of coastlines—in the case of New Zealand, some 11,000 km—drone-based imaging will provide a cost-effective and comprehensive quantitative assessment tool to be used in the presence of ever-increasing impacts from land-use changes [41,42], natural upheavals, and climate change. We advocate for concurrent collection of validation points for key biogenic features using RTK-GPS (including GCPs) and aerial imagery to enable repeatable aerial surveys and robust estimates of classification algorithm accuracy. We posit that this methodology is superior to embedding transects into aerial imagery (e.g., [23]) for several reasons: (1) It better controls for biases in validation point density for different habitat classes; (2) it allows habitat classes to be sampled in areas where transects will not be moved by wave action during sampling; and (3) it requires less post-processing to create validation samples. The use of this framework has proved invaluable for a greater understanding of the true conservation status of key habitat-forming species increasingly threatened by natural and anthropogenic disturbances.