Using Deep Learning to Count Albatrosses from Space: Assessing Results in Light of Ground Truth Uncertainty
Abstract
:1. Introduction
2. Materials and Methods
2.1. Imagery
2.2. Manual Counts
2.3. Detection Network
2.3.1. CNN overview
2.3.2. Network Architecture
2.3.3. Dataset Processing
2.3.4. Training Parameters
3. Results
3.1. Manual Counts
3.2. Network Results
3.2.1. Altering Assessment Ground Truth
3.2.2. Altering Training Ground Truth
4. Discussion
4.1. Manual Counts
4.2. Network Performance
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Abbreviations
VHR | Very high resolution |
WV-3 | WorldView-3 |
CNN | Convolutional neural network |
AvP | Average precision |
mAP | Mean average precision |
References
- Pettorelli, N.; Laurance, W.F.; O’Brien, T.G.; Wegmann, M.; Nagendra, H.; Turner, W. Satellite remote sensing for applied ecologists: Opportunities and challenges. J. Appl. Ecol. 2014, 51, 839–848. [Google Scholar] [CrossRef]
- Hollings, T.; Burgman, M.; van Andel, M.; Gilbert, M.; Robinson, T.; Robinson, A. How do you find the green sheep? A critical review of the use of remotely sensed imagery to detect and count animals. Methods Ecol. Evol. 2018. [Google Scholar] [CrossRef]
- Witmer, G.W. Wildlife population monitoring: Some practical considerations. Wildl. Res. 2005, 32, 259–263. [Google Scholar] [CrossRef]
- Fretwell, P.T.; Staniland, I.J.; Forcada, J. Whales from space: Counting southern right whales by satellite. PLoS ONE 2014, 9, e88655. [Google Scholar] [CrossRef] [Green Version]
- Oishi, Y.; Matsunaga, T. Support system for surveying moving wild animals in the snow using aerial remote-sensing images. Int. J. Remote Sens. 2014, 35, 1374–1394. [Google Scholar] [CrossRef]
- Anderson, K.; Gaston, K.J. Lightweight unmanned aerial vehicles will revolutionize spatial ecology. Front. Ecol. Environ. 2013, 11, 138–146. [Google Scholar] [CrossRef] [Green Version]
- Rowcliffe, J.M.; Carbone, C. Surveys using camera traps: Are we looking to a brighter future? Anim. Conserv. 2008, 11, 185–186. [Google Scholar] [CrossRef]
- LaRue, M.A.; Stapleton, S.; Anderson, M. Feasibility of using high-resolution satellite imagery to assess vertebrate wildlife populations. Conserv. Biol. 2017, 31, 213–220. [Google Scholar] [CrossRef]
- Xue, Y.; Wang, T.; Skidmore, A.K. Automatic counting of large mammals from very high resolution panchromatic satellite imagery. Remote Sens. 2017, 9, 878. [Google Scholar] [CrossRef] [Green Version]
- LaRue, M.A.; Stapleton, S. Estimating the abundance of polar bears on Wrangel Island during late summer using high-resolution satellite imagery: A pilot study. Polar Biol. 2018, 41, 2621–2626. [Google Scholar] [CrossRef]
- Borowicz, A.; Le, H.; Humphries, G.; Nehls, G.; Höschle, C.; Kosarev, V.; Lynch, H.J. Aerial-trained deep learning networks for surveying cetaceans from satellite imagery. PLoS ONE 2019, 14, e0212532. [Google Scholar] [CrossRef] [PubMed]
- Wang, D.; Shao, Q.; Yue, H. Surveying Wild Animals from Satellites, Manned Aircraft and Unmanned Aerial Systems (UASs): A Review. Remote Sens. 2019, 11, 1308. [Google Scholar] [CrossRef] [Green Version]
- Larue, M.A.; Stapleton, S.; Porter, C.; Atkinson, S.; Atwood, T.; Dyck, M.; Lecomte, N. Testing methods for using high-resolution satellite imagery to monitor polar bear abundance and distribution. Wildl. Soc. Bull. 2015, 39, 772–779. [Google Scholar] [CrossRef]
- Fretwell, P.T.; Scofield, P.; Phillips, R.A. Using super-high resolution satellite imagery to census threatened albatrosses. Ibis 2017, 159, 481–490. [Google Scholar] [CrossRef]
- Weinstein, B.G. A computer vision for animal ecology. J. Animal Ecol. 2017. [Google Scholar] [CrossRef]
- LeCun, Y.A.; Bengio, Y.; Hinton, G.E. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012, 1097–1105. [Google Scholar] [CrossRef]
- Gray, P.C.; Fleishman, A.B.; Klein, D.J.; McKown, M.W.; Bézy, V.S.; Lohmann, K.J.; Johnston, D.W. A convolutional neural network for detecting sea turtles in drone imagery. Methods Ecol. Evol. 2019, 10, 345–355. [Google Scholar] [CrossRef]
- Norouzzadeh, M.S.; Nguyen, A.; Kosmala, M.; Swanson, A.; Palmer, M.S.; Packer, C.; Clune, J. Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. Proc. Natl. Acad. Sci. USA 2018, 115, E5716–E5725. [Google Scholar] [CrossRef] [Green Version]
- Hänsch, R.; Hellwich, O. The Truth About Ground Truth: Label Noise in Human-Generated Reference Data. In Proceedings of the IGARSS 2019—2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 28 July–2 August 2019; pp. 5594–5597. [Google Scholar]
- Weimerskirch, H.; Delord, K.; Barbraud, C.; Le Bouard, F.; Ryan, P.G.; Fretwell, P.; Marteau, C. Status and trends of albatrosses in the French Southern Territories, Western Indian Ocean. Polar Biol. 2018, 41, 1963–1972. [Google Scholar] [CrossRef]
- Phillips, R.; Gales, R.; Baker, G.; Double, M.; Favero, M.; Quintana, F.; Tasker, M.; Weimerskirch, H.; Uhart, M.; Wolfaardt, A. The conservation status and priorities for albatrosses and large petrels. Biol. Conserv. 2016, 201, 169–183. [Google Scholar] [CrossRef]
- Fretwell, P.T.; LaRue, M.A.; Morin, P.; Kooyman, G.L.; Wienecke, B.; Ratcliffe, N.; Fox, A.J.; Fleming, A.H.; Porter, C.; Trathan, P.N. An emperor penguin population estimate: The first global, synoptic survey of a species from space. PLoS ONE 2012, 7, e33751. [Google Scholar] [CrossRef]
- LaRue, M.A.; Lynch, H.; Lyver, P.; Barton, K.; Ainley, D.; Pollard, A.; Fraser, W.; Ballard, G. A method for estimating colony sizes of Adélie penguins using remote sensing imagery. Polar Biol. 2014, 37, 507–517. [Google Scholar] [CrossRef]
- Naveen, R.; Lynch, H.J.; Forrest, S.; Mueller, T.; Polito, M. First direct, site-wide penguin survey at Deception Island, Antarctica, suggests significant declines in breeding chinstrap penguins. Polar Biol. 2012, 35, 1879–1888. [Google Scholar] [CrossRef]
- Hughes, B.J.; Martin, G.R.; Reynolds, S.J. The use of Google EarthTM satellite imagery to detect the nests of masked boobies Sula dactylatra. Wildl. Biol. 2011, 17, 210–216. [Google Scholar] [CrossRef] [Green Version]
- Bowler, E.; Fretwell, P.T.; French, G.; Mackiewicz, M. Using Deep Learning to Count Albatrosses from Space. In Proceedings of the IGARSS 2019—2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 28 July–2 August 2019; pp. 10099–10102. [Google Scholar]
- BirdLife International. Species Factsheet: Diomedea Exulans. 2020. Available online: http://www.birdlife.org (accessed on 8 June 2020).
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Miccai 2015, 234–241. [Google Scholar] [CrossRef] [Green Version]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Forsyth, D.A.; Ponce, J. Computer Vision: A Modern Approach; Prentice Hall Professional Technical Reference: Upper Saddle River, NJ, USA, 2002. [Google Scholar]
- Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar]
- Liang, S.; Wang, J. Advanced Remote Sensing: Terrestrial Information Extraction and Applications; Academic Press: Cambridge, MA, USA, 2019. [Google Scholar]
- Guo, F.; Cai, Z.x.; Xie, B.; Tang, J. Review and prospect of image dehazing techniques. Jisuanji Yingyong/J. Comput. Appl. 2010, 30, 2417–2421. [Google Scholar] [CrossRef]
- Jiang, H.; Lu, N. Multi-scale residual convolutional neural network for haze removal of remote sensing images. Remote Sens. 2018, 10, 945. [Google Scholar] [CrossRef] [Green Version]
- Fredembach, C.; Süsstrunk, S. Colouring the near-infrared. Soc. Imag. Sci. Technol. 2008, 2008, 176–182. [Google Scholar]
- Lee, K.Y.; Sim, J.Y. Cloud Removal of Satellite Images Using Convolutional Neural Network With Reliable Cloudy Image Synthesis Model. In Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22–25 September 2019; pp. 3581–3585. [Google Scholar]
- Fretwell, P.T.; Jackson, J.A.; Encina, M.J.U.; Häussermann, V.; Alvarez, M.J.P.; Olavarría, C.; Gutstein, C.S. Using remote sensing to detect whale strandings in remote areas: The case of sei whales mass mortality in Chilean Patagonia. PLoS ONE 2019, 14, e0222498. [Google Scholar]
- Cubaynes, H.C.; Fretwell, P.T.; Bamford, C.; Gerrish, L.; Jackson, J.A. Whales from space: Four mysticete species described using new VHR satellite imagery. Mar. Mamm. Sci. 2019, 35, 466–491. [Google Scholar] [CrossRef]
- Rodrigues, F.; Pereira, F.C. Deep learning from crowds. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018. [Google Scholar]
- Chabot, D.; Bird, D.M. Evaluation of an off-the-shelf unmanned aircraft system for surveying flocks of geese. Waterbirds 2012, 35, 170–174. [Google Scholar] [CrossRef]
- Patterson, C.; Koski, W.; Pace, P.; McLuckie, B.; Bird, D.M. Evaluation of an unmanned aircraft system for detecting surrogate caribou targets in Labrador. J. Unmanned Veh. Syst. 2015, 4, 53–69. [Google Scholar] [CrossRef] [Green Version]
- Brack, I.V.; Kindel, A.; Oliveira, L.F.B. Detection errors in wildlife abundance estimates from Unmanned Aerial Systems (UAS) surveys: Synthesis, solutions, and challenges. Methods Ecol. Evol. 2018, 9, 1864–1873. [Google Scholar] [CrossRef]
ref_ob | ob1 | ob2 | ob3 | ob4 | ob5 | Mean ± std | % dev | |
---|---|---|---|---|---|---|---|---|
Bird Island | 985 | 994 | 763 | 792 | 846 | 612 | 832 ± 145 | 17 |
Annenkov | 161 | 155 | 116 | 177 | 174 | 120 | 151 ± 26 | 18 |
Apotres | 171 | 165 | 168 | 177 | 174 | 162 | 170 ± 6 | 3 |
Grande Coulee | 649 | 690 | 656 | 840 | 741 | 638 | 702 ± 77 | 11 |
(a) | BI: Av. F1 = 0.74 | (b) | AN: Av. F1 = 0.67 | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ref_ob | ob1 | ob2 | ob3 | ob4 | ob5 | ref_ob | ob1 | ob2 | ob3 | ob4 | ob5 | |||
ref_ob | 1.00 | 0.81 | 0.72 | 0.81 | 0.78 | 0.68 | ref_ob | 1.00 | 0.63 | 0.62 | 0.57 | 0.57 | 0.60 | |
ob1 | 0.81 | 1.00 | 0.70 | 0.79 | 0.75 | 0.69 | ob1 | 0.63 | 1.00 | 0.72 | 0.73 | 0.70 | 0.73 | |
ob2 | 0.72 | 0.70 | 1.00 | 0.74 | 0.70 | 0.66 | ob2 | 0.62 | 0.72 | 1.00 | 0.68 | 0.70 | 0.78 | |
ob3 | 0.81 | 0.79 | 0.74 | 1.00 | 0.78 | 0.73 | ob3 | 0.57 | 0.73 | 0.68 | 1.00 | 0.66 | 0.69 | |
ob4 | 0.78 | 0.75 | 0.70 | 0.78 | 1.00 | 0.70 | ob4 | 0.57 | 0.70 | 0.70 | 0.66 | 1.00 | 0.67 | |
ob5 | 0.68 | 0.69 | 0.66 | 0.73 | 0.70 | 1.00 | ob5 | 0.60 | 0.73 | 0.78 | 0.69 | 0.67 | 1.00 | |
mean | 0.76 | 0.75 | 0.70 | 0.77 | 0.74 | 0.69 | mean | 0.60 | 0.70 | 0.70 | 0.67 | 0.66 | 0.70 | |
(c) | AP: Av. F1 = 0.92 | (d) | GC: Av. F1 = 0.85 | |||||||||||
ref_ob | ob1 | ob2 | ob3 | ob4 | ob5 | ref_ob | ob1 | ob2 | ob3 | ob4 | ob5 | |||
ref_ob | 1.00 | 0.93 | 0.91 | 0.93 | 0.89 | 0.92 | ref_ob | 1.00 | 0.83 | 0.82 | 0.77 | 0.79 | 0.78 | |
ob1 | 0.93 | 1.00 | 0.95 | 0.94 | 0.93 | 0.95 | ob1 | 0.83 | 1.00 | 0.91 | 0.87 | 0.89 | 0.88 | |
ob2 | 0.91 | 0.95 | 1.00 | 0.91 | 0.91 | 0.93 | ob2 | 0.82 | 0.91 | 1.00 | 0.85 | 0.87 | 0.86 | |
ob3 | 0.93 | 0.94 | 0.91 | 1.00 | 0.92 | 0.93 | ob3 | 0.77 | 0.87 | 0.85 | 1.00 | 0.88 | 0.82 | |
ob4 | 0.89 | 0.93 | 0.91 | 0.92 | 1.00 | 0.91 | ob4 | 0.79 | 0.89 | 0.87 | 0.88 | 1.00 | 0.86 | |
ob5 | 0.92 | 0.95 | 0.93 | 0.93 | 0.91 | 1.00 | ob5 | 0.78 | 0.88 | 0.86 | 0.82 | 0.86 | 1.00 | |
mean | 0.92 | 0.94 | 0.92 | 0.93 | 0.91 | 0.93 | mean | 0.80 | 0.87 | 0.86 | 0.84 | 0.86 | 0.84 |
(a) | BI | (b) | AN | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
union | ≥ 2 | ≥ 3 | ≥ 4 | ≥ 5 | inter | union | ≥ 2 | ≥ 3 | ≥ 4 | ≥ 5 | inter | |||
ref_ob | 0.83 | 0.91 | 0.88 | 0.83 | 0.75 | 0.55 | ref_ob | 0.67 | 0.66 | 0.70 | 0.69 | 0.70 | 0.56 | |
ob1 | 0.83 | 0.87 | 0.87 | 0.82 | 0.75 | 0.54 | ob1 | 0.66 | 0.85 | 0.84 | 0.81 | 0.78 | 0.57 | |
ob2 | 0.71 | 0.77 | 0.79 | 0.80 | 0.78 | 0.65 | ob2 | 0.54 | 0.78 | 0.85 | 0.88 | 0.84 | 0.70 | |
ob3 | 0.72 | 0.85 | 0.89 | 0.89 | 0.83 | 0.64 | ob3 | 0.72 | 0.82 | 0.79 | 0.74 | 0.70 | 0.52 | |
ob4 | 0.75 | 0.84 | 0.85 | 0.84 | 0.79 | 0.61 | ob4 | 0.71 | 0.81 | 0.77 | 0.74 | 0.71 | 0.53 | |
ob5 | 0.61 | 0.73 | 0.78 | 0.80 | 0.82 | 0.75 | ob5 | 0.55 | 0.76 | 0.84 | 0.88 | 0.85 | 0.68 | |
mean | 0.74 | 0.83 | 0.84 | 0.83 | 0.78 | 0.62 | mean | 0.64 | 0.78 | 0.80 | 0.79 | 0.76 | 0.59 | |
(c) | AP | (d) | GC | |||||||||||
union | ≥2 | ≥3 | ≥4 | ≥5 | inter | union | ≥2 | ≥3 | ≥4 | ≥5 | inter | |||
ref_ob | 0.91 | 0.94 | 0.94 | 0.93 | 0.94 | 0.90 | ref_ob | 0.79 | 0.83 | 0.85 | 0.85 | 0.85 | 0.83 | |
ob1 | 0.89 | 0.96 | 0.98 | 0.98 | 0.97 | 0.92 | ob1 | 0.82 | 0.92 | 0.95 | 0.95 | 0.92 | 0.80 | |
ob2 | 0.90 | 0.95 | 0.94 | 0.96 | 0.95 | 0.91 | ob2 | 0.80 | 0.91 | 0.94 | 0.95 | 0.92 | 0.82 | |
ob3 | 0.92 | 0.96 | 0.96 | 0.95 | 0.94 | 0.88 | ob3 | 0.92 | 0.92 | 0.90 | 0.88 | 0.83 | 0.71 | |
ob4 | 0.92 | 0.95 | 0.94 | 0.93 | 0.92 | 0.89 | ob4 | 0.85 | 0.93 | 0.93 | 0.91 | 0.88 | 0.76 | |
ob5 | 0.88 | 0.93 | 0.95 | 0.96 | 0.97 | 0.93 | ob5 | 0.78 | 0.87 | 0.90 | 0.90 | 0.91 | 0.84 | |
mean | 0.91 | 0.95 | 0.95 | 0.95 | 0.95 | 0.90 | mean | 0.83 | 0.90 | 0.91 | 0.91 | 0.89 | 0.79 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bowler, E.; Fretwell, P.T.; French, G.; Mackiewicz, M. Using Deep Learning to Count Albatrosses from Space: Assessing Results in Light of Ground Truth Uncertainty. Remote Sens. 2020, 12, 2026. https://doi.org/10.3390/rs12122026
Bowler E, Fretwell PT, French G, Mackiewicz M. Using Deep Learning to Count Albatrosses from Space: Assessing Results in Light of Ground Truth Uncertainty. Remote Sensing. 2020; 12(12):2026. https://doi.org/10.3390/rs12122026
Chicago/Turabian StyleBowler, Ellen, Peter T. Fretwell, Geoffrey French, and Michal Mackiewicz. 2020. "Using Deep Learning to Count Albatrosses from Space: Assessing Results in Light of Ground Truth Uncertainty" Remote Sensing 12, no. 12: 2026. https://doi.org/10.3390/rs12122026