UAV-Based 3D-Calibration of Thermal Cameras for Bat Flight Monitoring in Large Outdoor Environments
Abstract
:1. Introduction
2. Related Work
- The calibration process takes at least two drone flights of a few minutes at the site and can, therefore, be practically used as a mobile setup at different outdoor environments.
- The method is not dependent on any other real-world objects (and knowledge about their dimensions) apart from the drone GPS data and is, therefore, not susceptible to any further errors from coordinate system transformations based on known landmarks.
- The method allows for large base distances, making it especially useful for low-resolution cameras outdoors.
- The 3D points are directly given in geographic coordinates (WGS 84). They can be directly projected to metric UTM coordinates (using the Python module utm [42]) and shifted to the turbine origin, making it possible to calculate the geographic direction, distance to the turbine, flight height and speed of the detected animals. Another advantage of geographic coordinates is that measurements from other camera setups at different positions can be related to each other.
3. Materials and Methods
3.1. Hardware and Spatial Arrangement
3.2. 3D Calibration and 3D Reconstruction
3.3. 3D Error
3.4. Time Synchronization of Camera and Drone Signal
3.5. Automated 2D Image Points Detection
3.6. Corresponding Image Points of Both Cameras
4. Results
5. Discussion
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
DLT | Direct Linear Transform |
GLONSASS | Global Navigation Satellite System |
GPS | Global Positioning System |
HFOV | Horizontal Field of View |
IMG | Image(s) |
NETD | Noise-Equivalent Temperature Difference |
RMSE | Root Mean Squared Error |
RTK | Real-time Kinematic Positioning |
UAV | Unmanned Aerial Vehicle |
UTM | Universal Transverse Mercator (earth coordinate system (projected, metric)) |
WGS 84 | World Geodetic System 1984 (earth coordinate system (ellipsoid)) |
Appendix A. Camera Inter-Axial Distance and Its Effects on the 3D Resolution
Appendix B. Single Camera Calibration
Appendix C. More Examples of 3D Flight Tracks
Appendix D. Example Images of Bats
Appendix E. Morphological Operations as Part of the 2D Detection Process
References
- European Commission. European Wind Power Action Plan. 2023. Available online: https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52023DC0669 (accessed on 2 October 2024).
- Arnett, E.B.; Baerwald, E.F. Impacts of wind energy development on bats: Implications for conservation. In Bat Evolution, Ecology, and Conservation; Springer: Berlin/Heidelberg, Germany, 2013; pp. 435–456. [Google Scholar] [CrossRef]
- Kunz, T.H.; Arnett, E.B.; Erickson, W.P.; Hoar, A.R.; Johnson, G.D.; Larkin, R.P.; Strickland, M.D.; Thresher, R.W.; Tuttle, M.D. Ecological impacts of wind energy development on bats: Questions, research needs, and hypotheses. Front. Ecol. Environ. 2007, 5, 315–324. [Google Scholar] [CrossRef]
- Göttert, T.; Starik, N. Human–wildlife conflicts across landscapes—General applicability vs. case specificity. Diversity 2022, 14, 380. [Google Scholar] [CrossRef]
- Baerwald, E.F.; D’Amours, G.H.; Klug, B.J.; Barclay, R.M. Barotrauma is a significant cause of bat fatalities at wind turbines. Curr. Biol. 2008, 18, R695–R696. [Google Scholar] [CrossRef] [PubMed]
- Guest, E.E.; Stamps, B.F.; Durish, N.D.; Hale, A.M.; Hein, C.D.; Morton, B.P.; Weaver, S.P.; Fritts, S.R. An updated review of hypotheses regarding bat attraction to wind turbines. Animals 2022, 12, 343. [Google Scholar] [CrossRef] [PubMed]
- Cryan, P.M. Mating behavior as a possible cause of bat fatalities at wind turbines. J. Wildl. Manag. 2008, 72, 845–849. [Google Scholar] [CrossRef]
- Rydell, J.; Bach, L.; Dubourg-Savage, M.J.; Green, M.; Rodrigues, L.; Hedenström, A. Mortality of bats at wind turbines links to nocturnal insect migration? Eur. J. Wildl. Res. 2010, 56, 823–827. [Google Scholar] [CrossRef]
- Cryan, P.M.; Gorresen, P.M.; Hein, C.D.; Schirmacher, M.R.; Diehl, R.H.; Huso, M.M.; Hayman, D.T.; Fricker, P.D.; Bonaccorso, F.J.; Johnson, D.H.; et al. Behavior of bats at wind turbines. Proc. Natl. Acad. Sci. USA 2014, 111, 15126–15131. [Google Scholar] [CrossRef] [PubMed]
- Reusch, C.; Lozar, M.; Kramer-Schadt, S.; Voigt, C.C. Coastal onshore wind turbines lead to habitat loss for bats in Northern Germany. J. Environ. Manag. 2022, 310, 114715. [Google Scholar] [CrossRef]
- Voigt, C.C.; Russo, D.; Runkel, V.; Goerlitz, H.R. Limitations of acoustic monitoring at wind turbines to evaluate fatality risk of bats. Mammal Rev. 2021, 51, 559–570. [Google Scholar] [CrossRef]
- Voigt, C.C.; Scherer, C.; Runkel, V. Modeling the power of acoustic monitoring to predict bat fatalities at wind turbines. Conserv. Sci. Pract. 2022, 4, e12841. [Google Scholar] [CrossRef]
- Horn, J.W.; Arnett, E.B.; Kunz, T.H. Behavioral responses of bats to operating wind turbines. J. Wildl. Manag. 2008, 72, 123–132. [Google Scholar] [CrossRef]
- Betke, M.; Hirsh, D.E.; Makris, N.C.; McCracken, G.F.; Procopio, M.; Hristov, N.I.; Tang, S.; Bagchi, A.; Reichard, J.D.; Horn, J.W.; et al. Thermal imaging reveals significantly smaller Brazilian free-tailed bat colonies than previously estimated. J. Mammal. 2008, 89, 18–24. [Google Scholar] [CrossRef]
- Bentley, I.; Kuczynska, V.; Eddington, V.M.; Armstrong, M.; Kloepper, L.N. BatCount: A software program to count moving animals. PLoS ONE 2023, 18, e0278012. [Google Scholar] [CrossRef] [PubMed]
- Bentley, I.; Gebran, M.; Vorderer, S.; Ralston, J.; Kloepper, L. Utilizing Neural Networks to Resolve Individual Bats and Improve Automated Counts. In Proceedings of the 2023 IEEE World AI IoT Congress (AIIoT), Virtual, 7–10 June 2023; IEEE: Piscataway Township, NJ, USA, 2023; pp. 0112–0119. [Google Scholar]
- Håkansson, J.; Quinn, B.L.; Shultz, A.L.; Swartz, S.M.; Corcoran, A.J. Application of a novel deep learning–based 3D videography workflow to bat flight. Ann. N. Y. Acad. Sci. 2024, 1536, 92–106. [Google Scholar] [CrossRef] [PubMed]
- Darras, K.; Yusti, E.; Knorr, A.; Huang, J.C.C.; Kartono, A.P. Sampling flying bats with thermal and near-infrared imaging and ultrasound recording: Hardware and workflow for bat point counts. F1000Research 2022, 10, 189. [Google Scholar] [CrossRef] [PubMed]
- Hochradel, K.; Häcker, T.; Hohler, T.; Becher, A.; Wildermann, S.; Sutor, A. Three-Dimensional Localization of Bats: Visual and Acoustical. IEEE Sensors J. 2019, 19, 5825–5833. [Google Scholar] [CrossRef]
- Robinson Willmott, J.; Forcey, G.; Vukovich, M. New Insights into the Influence of Turbines on the Behaviour of Migrant Birds: Implications for Predicting Impacts of Offshore Wind Developments on Wildlife. J. Phys. Conf. Ser. 2023, 2507, 012006. [Google Scholar] [CrossRef]
- Bottalico, F.; Niezrecki, C.; Jerath, K.; Luo, Y.; Sabato, A. Sensor-Based Calibration of Camera’s Extrinsic Parameters for Stereophotogrammetry. IEEE Sensors J. 2023, 23, 7776–7785. [Google Scholar] [CrossRef]
- Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1330–1334. [Google Scholar] [CrossRef]
- Li, H.; Wang, S.; Bai, Z.; Wang, H.; Li, S.; Wen, S. Research on 3D reconstruction of binocular vision based on thermal infrared. Sensors 2023, 23, 7372. [Google Scholar] [CrossRef] [PubMed]
- Liu, X.; Tian, J.; Kuang, H.; Ma, X. A stereo calibration method of multi-camera based on circular calibration board. Electronics 2022, 11, 627. [Google Scholar] [CrossRef]
- Zheng, H.; Duan, F.; Li, T.; Li, J.; Niu, G.; Cheng, Z.; Li, X. A Stable, Efficient, and High-Precision Non-Coplanar Calibration Method: Applied for Multi-Camera-Based Stereo Vision Measurements. Sensors 2023, 23, 8466. [Google Scholar] [CrossRef]
- Hochradel, K.; Hartmann, S.; Reers, H.; Luedtke, B.; Schauer-Weisshahn, H.; Thomsen, K.M.; Hoetker, H.; Brinkmann, R. Three-dimensional analysis of bat flight paths around small wind turbines suggests no major collision risk or behavioral changes. Mammal Res. 2022, 67, 83–98. [Google Scholar] [CrossRef]
- Matzner, S.; Warfel, T.; Hull, R. ThermalTracker-3D: A thermal stereo vision system for quantifying bird and bat activity at offshore wind energy sites. Ecol. Inform. 2020, 57, 101069. [Google Scholar] [CrossRef]
- Matzner, S.; Warfel, T.; Hull, R.; Williams, N. ThermalTracker-3D Offshore Validation Technical Report; Technical Report; Pacific Northwest National Laboratory (PNNL): Richland, DC, USA, 2022. [Google Scholar]
- Schneider, S.R.; Kramer, S.H.; Bernstein, S.B.; Terrill, S.B.; Ainley, D.G.; Matzner, S. Autonomous thermal tracking reveals spatiotemporal patterns of seabird activity relevant to interactions with floating offshore wind facilities. Front. Mar. Sci. 2024, 11, 1346758. [Google Scholar] [CrossRef]
- Son, J.Y.; Chernyshov, O.; Lee, C.H.; Park, M.C.; Yano, S. Depth resolution in three-dimensional images. JOSA A 2013, 30, 1030–1038. [Google Scholar] [CrossRef]
- Li, G.; Xu, Z.; Zhang, Y.; Xin, C.; Wang, J.; Yan, S. Calibration method for binocular vision system with large field of view based on small target image splicing. Meas. Sci. Technol. 2024, 35, 085006. [Google Scholar] [CrossRef]
- Gao, Z.; Gao, Y.; Su, Y.; Liu, Y.; Fang, Z.; Wang, Y.; Zhang, Q. Stereo camera calibration for large field of view digital image correlation using zoom lens. Measurement 2021, 185, 109999. [Google Scholar] [CrossRef]
- Corcoran, A.J.; Schirmacher, M.R.; Black, E.; Hedrick, T.L. ThruTracker: Open-source software for 2-D and 3-D animal video tracking. bioRxiv 2021, 2021-05. [Google Scholar] [CrossRef]
- Theriault, D.H.; Fuller, N.W.; Jackson, B.E.; Bluhm, E.; Evangelista, D.; Wu, Z.; Betke, M.; Hedrick, T.L. A protocol and calibration method for accurate multi-camera field videography. J. Exp. Biol. 2014, 217, 1843–1848. [Google Scholar] [CrossRef] [PubMed]
- Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Lourakis, M.I.; Argyros, A.A. SBA: A software package for generic sparse bundle adjustment. ACM Trans. Math. Softw. (TOMS) 2009, 36, 1–30. [Google Scholar] [CrossRef]
- Happ, C.; Hochradel, K.; Sutor, A. Comparison of two 3D calibration methods for thermal imaging cameras to track bat flight paths near wind turbines. Tm-Tech. Mess. 2024, 91, 61–65. [Google Scholar] [CrossRef]
- Tripicchio, P.; D’Avella, S.; Camacho-Gonzalez, G.; Landolfi, L.; Baris, G.; Avizzano, C.A.; Filippeschi, A. Multi-Camera Extrinsic Calibration for Real-Time Tracking in Large Outdoor Environments. J. Sens. Actuator Netw. 2022, 11, 40. [Google Scholar] [CrossRef]
- Feng, W.; Zhang, S.; Liu, H.; Yu, Q.; Wu, S.; Zhang, D. Unmanned aerial vehicle-aided stereo camera calibration for outdoor applications. Opt. Eng. 2020, 59, 014110. [Google Scholar] [CrossRef]
- Fedorov, I.; Thörnberg, B.; Alqaysi, H.; Lawal, N.; O’Nils, M. A Two-Layer 3-D Reconstruction Method and Calibration for Multicamera-Based Volumetric Positioning and Characterization. IEEE Trans. Instrum. Meas. 2021, 70, 1–9. [Google Scholar] [CrossRef]
- Zhang, Y.; Yang, J.; Li, G.; Zhao, T.; Song, X.; Zhang, S.; Li, A.; Bian, H.; Li, J.; Zhang, M. Camera Calibration for Long-Distance Photogrammetry Using Unmanned Aerial Vehicles. J. Sens. 2022, 2022, 8573315. [Google Scholar] [CrossRef]
- van Andel, B.; Tobias Bieniek, T.I.B. utm Python Module. [Code]. 2012. Available online: https://github.com/Turbo87/utm (accessed on 2 October 2024).
- Happ, C. Batflight 3D. [Code, Dataset]. 2024. Available online: https://github.com/christofhapp/batflight3D_publication/tree/v1.0.1 (accessed on 2 October 2024).
- Haralick, R.M.; Lee, C.n.; Ottenburg, K.; Nölle, M. Analysis and solutions of the three point perspective pose estimation problem. In Proceedings of the CVPR, Maui, HI, USA, 3–6 June 1991; Volume 91, pp. 592–598. [Google Scholar]
- Quan, L.; Lan, Z. Linear n-point camera pose determination. IEEE Trans. Pattern Anal. Mach. Intell. 1999, 21, 774–780. [Google Scholar] [CrossRef]
- Terzakis, G.; Lourakis, M. A consistently fast and globally optimal solution to the perspective-n-point problem. In Proceedings of the European Conference on Computer Vision, Glasgow, UK, 23–28 August 2020; Springer: Berlin/Heidelberg, Germany, 2020; pp. 478–494. [Google Scholar] [CrossRef]
- Van der Walt, S.; Schönberger, J.L.; Nunez-Iglesias, J.; Boulogne, F.; Warner, J.D.; Yager, N.; Gouillart, E.; Yu, T. scikit-image: Image processing in Python. PeerJ 2014, 2, e453. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Z. Determining the epipolar geometry and its uncertainty: A review. Int. J. Comput. Vis. 1998, 27, 161–195. [Google Scholar] [CrossRef]
Method | Error Optimization | Primary Coordinate System | Calibration Object | Inter-Axial Camera Distance | Cameras | Purpose |
---|---|---|---|---|---|---|
Hochradel et al. [26] Matzner et al. [27] | 2D error | camera | thermal checkerboard | small | thermal | detecting bats |
Corcoran et al. [33] | 2D error | camera | natural landmarks | large | thermal | detecting bats |
Tripichio et al. [38] | 2D error | camera | wand points (only optical) | large | visual | tracking in general |
Feng et al. [39] | 2D error | camera | drone points (only optical) | large | visual | not explained |
Fedorov et al. [40] | 2D error | WGS 84 | ground GPS | large | visual | detecting birds (daylight) |
Zhang et al. [41] | 2D error | WGS 84 | drone GPS points | large | visual | not explained |
Proposed Method | 3D error | WGS 84 | drone GPS flight track | large | thermal | detecting nightly bats and birds |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Happ, C.; Sutor, A.; Hochradel, K. UAV-Based 3D-Calibration of Thermal Cameras for Bat Flight Monitoring in Large Outdoor Environments. Remote Sens. 2024, 16, 4682. https://doi.org/10.3390/rs16244682
Happ C, Sutor A, Hochradel K. UAV-Based 3D-Calibration of Thermal Cameras for Bat Flight Monitoring in Large Outdoor Environments. Remote Sensing. 2024; 16(24):4682. https://doi.org/10.3390/rs16244682
Chicago/Turabian StyleHapp, Christof, Alexander Sutor, and Klaus Hochradel. 2024. "UAV-Based 3D-Calibration of Thermal Cameras for Bat Flight Monitoring in Large Outdoor Environments" Remote Sensing 16, no. 24: 4682. https://doi.org/10.3390/rs16244682
APA StyleHapp, C., Sutor, A., & Hochradel, K. (2024). UAV-Based 3D-Calibration of Thermal Cameras for Bat Flight Monitoring in Large Outdoor Environments. Remote Sensing, 16(24), 4682. https://doi.org/10.3390/rs16244682