Performance Comparison of Feature Generation Algorithms for Mosaic Photoacoustic Microscopy
Abstract
:1. Introduction
2. Materials and Methods
2.1. Feature Generation Process
2.2. Scale Invariant Feature Transform (SIFT)
2.3. Speeded-Up Robust Features (SURF)
2.4. Oriented FAST and Rotated BRIEF (ORB)
2.5. Accelerated-KAZE (AKAZE)
2.6. GoodPoint
2.7. Optical-Resolution Photoacoustic Microscopy (PAM)
2.8. Animal Preparing
2.9. Mosaic PAM Imaging Process
2.10. Computing System
2.11. Performance Evaluation
- (1)
- Processing time: Using system resources directly, we estimated the time consumed to complete the feature generation process. The processing time began from algorithm detection until the matching process was completed. By estimating the processing time, we could determine which methods were lighter and required few resources to process while maintaining the same performance.
- (2)
- Number of matching points: By maintaining the same conditions for the mosaic imaging generator (), each feature generation method provided a different number of matching points. The number of matching points (matrix transformation conditions) should be as more than four as possible. More matching points mean more chances for a high-quality mosaicking process.
- (3)
- Good matching ratio (GMR): By removing negative matching points () and maintaining positive matching points (), we estimated the accuracy of the feature generation methods.
- (4)
- Matching efficiency: Matching efficiency shows the number of positive matching points in one millisecond. In this way, we estimated algorithms to reduce the time consumed while maintaining similar performance.
2.12. Dataset Preparing
3. Results
3.1. Performance Comparison
3.2. Whole Mosaic OR-PAM Image Generation
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Yun Tian, G.; Gledhill, D.; Taylor, D. Comprehensive Interest Points Based Imaging Mosaic. Pattern Recognit. Lett. 2003, 24, 1171–1179. [Google Scholar] [CrossRef]
- Can, A.; Stewart, C.V.; Roysam, B.; Tanenbaum, H.L. A Feature-Based Technique for Joint, Linear Estimation of High-Order Image-to-Mosaic Transformations: Mosaicing the Curved Human Retina. IEEE Trans. Pattern Anal. Mach. Intell. 2002, 24, 412–419. [Google Scholar] [CrossRef]
- Battiato, S.; Di Blasi, G.; Farinella, G.M.; Gallo, G. Digital Mosaic Frameworks-An Overview. Comput. Graph. Forum 2007, 26, 794–812. [Google Scholar] [CrossRef] [Green Version]
- Li, X.; Feng, R.; Guan, X.; Shen, H.; Zhang, L. Remote Sensing Image Mosaicking: Achievements and Challenges. IEEE Geosci. Remote Sens. Mag. 2019, 7, 8–22. [Google Scholar] [CrossRef]
- Katz, D.S.; Berriman, G.B.; Mann, R.G. Collaborative Astronomical Image Mosaics. arXiv 2010, arXiv:10115294. [Google Scholar]
- Wu, M.; Yang, C.; Song, X.; Hoffmann, W.C.; Huang, W.; Niu, Z.; Wang, C.; Li, W. Evaluation of Orthomosics and Digital Surface Models Derived from Aerial Imagery for Crop Type Mapping. Remote Sens. 2017, 9, 239. [Google Scholar] [CrossRef] [Green Version]
- Turner, D.; Lucieer, A.; Watson, C. An Automated Technique for Generating Georectified Mosaics from Ultra-High Resolution Unmanned Aerial Vehicle (UAV) Imagery, Based on Structure from Motion (SfM) Point Clouds. Remote Sens. 2012, 4, 1392–1410. [Google Scholar] [CrossRef] [Green Version]
- Zhang, W.; Li, X.; Yu, J.; Kumar, M.; Mao, Y. Remote Sensing Image Mosaic Technology Based on SURF Algorithm in Agriculture. EURASIP J. Image Video Process. 2018, 2018, 1–9. [Google Scholar] [CrossRef]
- Li, Z.; Isler, V. Large Scale Image Mosaic Construction for Agricultural Applications. IEEE Robot. Autom. Lett. 2016, 1, 295–302. [Google Scholar] [CrossRef]
- Díaz, M.; de Moura, J.; Novo, J.; Ortega, M. Automatic Wide Field Registration and Mosaicking of OCTA Images Using Vascularity Information. Procedia Comput. Sci. 2019, 159, 505–513. [Google Scholar] [CrossRef]
- Chow, S.K.; Hakozaki, H.; Price, D.L.; MacLean, N.a.B.; Deerinck, T.J.; Bouwer, J.C.; Martone, M.E.; Peltier, S.T.; Ellisman, M.H. Automated Microscopy System for Mosaic Acquisition and Processing. J. Microsc. 2006, 222, 76–84. [Google Scholar] [CrossRef]
- Mokso, R. X-Ray Mosaic Nanotomography of Large Microorganisms. J. Struct. Biol. 2012, 177, 233–238. [Google Scholar] [CrossRef]
- Piccinini, F.; Bevilacqua, A.; Lucarelli, E. Automated Image Mosaics by Non-Automated Light Microscopes: The MicroMos Software Tool: Automated Image Mosaics by Non-Automated Light Microscopes. J. Microsc. 2013, 252, 226–250. [Google Scholar] [CrossRef]
- Harris, C.; Stephens, M. A Combined Corner and Edge Detector. In Alvey Vision Conference; Alvey Vision Club: Manchester, UK, 1988; pp. 23.1–23.6. [Google Scholar]
- Rosten, E.; Drummond, T. Machine Learning for High-Speed Corner Detection. In Computer Vision–ECCV 2006; Leonardis, A., Bischof, H., Pinz, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 430–443. [Google Scholar]
- Hutchison, D.; Kanade, T.; Kittler, J.; Kleinberg, J.M.; Mattern, F.; Mitchell, J.C.; Naor, M.; Nierstrasz, O.; Pandu Rangan, C.; Steffen, B.; et al. BRIEF: Binary Robust Independent Elementary Features. In Computer Vision–ECCV 2010; Daniilidis, K., Maragos, P., Paragios, N., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2010; Volume 6314, pp. 778–792. ISBN 978-3-642-15560-4. [Google Scholar]
- Zhao, H.; Chen, N.; Li, T.; Zhang, J.; Lin, R.; Gong, X.; Song, L.; Liu, Z.; Liu, C.; Hidalgo, F.; et al. ORB: An Efficient Alternative to SIFT or SURF. Int. J. Comput. Vis. 2019, 4, 4162–4169. [Google Scholar] [CrossRef]
- Kalms, L.; Mohamed, K.; Göhringer, D. Accelerated Embedded AKAZE Feature Detection Algorithm on FPGA. In Proceedings of the 8th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies, Bochum, Germany, 7–9 June 2017. [Google Scholar] [CrossRef]
- Belikov, A.V.; Potapov, A.S.; Yashchenko, A.V. Goodpoint: Unsupervised Learning of Key Point Detection and Description. Sci. Tech. J. Inf. Technol. Mech. Opt. 2021, 21, 92–101. [Google Scholar] [CrossRef]
- Zhu, H.; Wen, X.; Zhang, F.; Wang, X.; Wang, G. Homography Estimation Based on Order-Preserving Constraint and Similarity Measurement. IEEE Access 2018, 6, 28680–28690. [Google Scholar] [CrossRef]
- Wang, L.; Traub, J.; Heining, S.M.; Benhimane, S.; Euler, E.; Graumann, R.; Navab, N. Long Bone X-Ray Image Stitching Using C-Arm Motion Estimation. Inform. Aktuell 2009, 2009, 202–206. [Google Scholar] [CrossRef] [Green Version]
- Meine, H.; Hering, A. Efficient Prealignment of CT Scans for Registration through a Bodypart Regressor. arXiv 2019, arXiv:190908898. [Google Scholar]
- Shilling, R.Z.; Brummer, M.E.; Mewes, K. Merging Multiple Stacks MRI into a Single Data Volume. In Proceedings of the 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, Arlington, VA, USA, 6–9 April 2006; pp. 1012–1015. [Google Scholar]
- Townsend, D.W. Combined Positron Emission Tomography-Computed Tomography: The Historical Perspective. Semin. Ultrasound CT MRI 2008, 29, 232–235. [Google Scholar] [CrossRef] [Green Version]
- Yaniv, Z.; Joskowicz, L. Long Bone Panoramas from Fluoroscopic X-Ray Images. IEEE Trans. Med. Imaging 2004, 23, 26–35. [Google Scholar] [CrossRef]
- Bakar, S.A.; Jiang, X.; Gui, X.; Li, G.; Li, Z. Image Stitching for Chest Digital Radiography Using the SIFT and SURF Feature Extraction by RANSAC Algorithm. J. Phys. Conf. Ser. 2020, 1624. [Google Scholar] [CrossRef]
- Ni, D.; Chui, Y.P.; Qu, Y.; Yang, X.; Qin, J.; Wong, T.-T.; Ho, S.S.H.; Heng, P.A. Reconstruction of Volumetric Ultrasound Panorama Based on Improved 3D SIFT. Comput. Med. Imaging Graph. 2009, 33, 559–566. [Google Scholar] [CrossRef]
- Seo, J.-H.; Yang, S.; Kang, M.-S.; Her, N.-G.; Nam, D.-H.; Choi, J.-H.; Kim, M.H. Automated Stitching of Microscope Images of Fluorescence in Cells with Minimal Overlap. Micron 2019, 126, 102718. [Google Scholar] [CrossRef]
- Jain, M.; Rajadhyaksha, M.; Nehal, K. Implementation of Fluorescence Confocal Mosaicking Microscopy by “Early Adopter” Mohs Surgeons and Dermatologists: Recent Progress. J. Biomed. Opt. 2017, 22, 17. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Zhang, H.F.; Maslov, K.; Stoica, G.; Wang, L.V. Functional Photoacoustic Microscopy for High-Resolution and Noninvasive in Vivo Imaging. Nat. Biotechnol. 2006, 24, 848–851. [Google Scholar] [CrossRef] [PubMed]
- Lee, C.; Kim, J.Y.; Kim, C. Recent Progress on Photoacoustic Imaging Enhanced with Microelectromechanical Systems (MEMS) Technologies. Micromachines 2018, 9, 584. [Google Scholar] [CrossRef] [Green Version]
- Jung, D.; Park, S.; Lee, C.; Kim, H. Recent Progress on Near-Infrared Photoacoustic Imaging: Imaging Modality and Organic Semiconducting Agents. Polymers 2019, 11, 1693. [Google Scholar] [CrossRef] [Green Version]
- Jeon, S.; Kim, J.; Lee, D.; Baik, J.W.; Kim, C. Review on Practical Photoacoustic Microscopy. Photoacoustics 2019, 15, 100141. [Google Scholar] [CrossRef]
- Strohm, E.M.; Moore, M.J.; Kolios, M.C. Single Cell Photoacoustic Microscopy: A Review. IEEE J. Sel. Top. Quantum Electron. 2016, 22, 137–151. [Google Scholar] [CrossRef]
- Hai, P.; Imai, T.; Xu, S.; Zhang, R.; Aft, R.L.; Zou, J.; Wang, L.V. High-Throughput, Label-Free, Single-Cell Photoacoustic Microscopy of Intratumoral Metabolic Heterogeneity. Nat. Biomed. Eng. 2019, 3, 381–391. [Google Scholar] [CrossRef]
- Han, J.; Yang, P.; Tang, S. Local Acoustic Field Enhancement of Single Cell Photoacoustic Signal Detection Based on Metamaterial Structure. AIP Adv. 2019, 9, 095064. [Google Scholar] [CrossRef] [Green Version]
- Lee, C.; Jeon, M.; Jeon, M.Y.; Kim, J.; Kim, C. In Vitro Photoacoustic Measurement of Hemoglobin Oxygen Saturation Using a Single Pulsed Broadband Supercontinuum Laser Source. Appl. Opt. 2014, 53, 3884–3889. [Google Scholar] [CrossRef] [PubMed]
- Zhou, H.-C.; Chen, N.; Zhao, H.; Yin, T.; Zhang, J.; Zheng, W.; Song, L.; Liu, C.; Zheng, R. Optical-Resolution Photoacoustic Microscopy for Monitoring Vascular Normalization during Anti-Angiogenic Therapy. Photoacoustics 2019, 15, 100143. [Google Scholar] [CrossRef]
- Zhao, J.; Zhao, Q.; Lin, R.; Meng, J. A Microvascular Image Analysis Method for Optical-Resolution Photoacoustic Microscopy. J. Innov. Opt. Health Sci. 2020, 13, 2050019. [Google Scholar] [CrossRef]
- Mai, T.T.; Vo, M.-C.; Chu, T.-H.; Kim, J.Y.; Kim, C.; Lee, J.-J.; Jung, S.-H.; Lee, C. Pilot Study: Quantitative Photoacoustic Evaluation of Peripheral Vascular Dynamics Induced by Carfilzomib In Vivo. Sensors 2021, 21, 836. [Google Scholar] [CrossRef]
- Mai, T.T.; Yoo, S.W.; Park, S.; Kim, J.Y.; Choi, K.-H.; Kim, C.; Kwon, S.Y.; Min, J.-J.; Lee, C. In Vivo Quantitative Vasculature Segmentation and Assessment for Photodynamic Therapy Process Monitoring Using Photoacoustic Microscopy. Sensors 2021, 21, 1776. [Google Scholar] [CrossRef]
- Wong, T.T.W.; Zhang, R.; Zhang, C.; Hsu, H.-C.; Maslov, K.I.; Wang, L.; Shi, J.; Chen, R.; Shung, K.K.; Zhou, Q.; et al. Label-Free Automated Three-Dimensional Imaging of Whole Organs by Microtomy-Assisted Photoacoustic Microscopy. Nat. Commun. 2017, 8, 1–8. [Google Scholar] [CrossRef]
- Park, E.-Y.; Lee, D.; Lee, C.; Kim, C. Non-Ionizing Label-Free Photoacoustic Imaging of Bones. IEEE Access 2020, 8, 160915–160920. [Google Scholar] [CrossRef]
- Bi, R.; Ma, Q.; Mo, H.; Olivo, M.; Pu, Y. Optical-resolution photoacoustic microscopy of brain vascular imaging in small animal tumor model using nanosecond solid-state laser. In Neurophotonics and Biomedical Spectroscopy; Elsevier: Amsterdam, The Netherlands, 2018; pp. 159–187. ISBN 978-0-323-48067-3. [Google Scholar]
- Yao, J.; Wang, L.; Yang, J.-M.; Maslov, K.I.; Wong, T.T.W.; Li, L.; Huang, C.-H.; Zou, J.; Wang, L.V. High-Speed Label-Free Functional Photoacoustic Microscopy of Mouse Brain in Action. Nat. Methods 2015, 12, 407–410. [Google Scholar] [CrossRef]
- Kim, J.; Kim, J.Y.; Jeon, S.; Baik, J.W.; Cho, S.H.; Kim, C. Super-Resolution Localization Photoacoustic Microscopy Using Intrinsic Red Blood Cells as Contrast Absorbers. Light Sci. Appl. 2019, 8, 103. [Google Scholar] [CrossRef]
- Yeh, C.; Hu, S.; Maslov, K.; Wang, L.V. Photoacoustic Microscopy of Blood Pulse Wave. J. Biomed. Opt. 2012, 17, 070504. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Liu, W.; Shcherbakova, D.M.; Kurupassery, N.; Li, Y.; Zhou, Q.; Verkhusha, V.V.; Yao, J. Quad-Mode Functional and Molecular Photoacoustic Microscopy. Sci. Rep. 2018, 8, 11123. [Google Scholar] [CrossRef] [PubMed]
- Yao, J.; Wang, L.V. Recent Progress in Photoacoustic Molecular Imaging. Curr. Opin. Chem. Biol. 2018, 45, 104–112. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Yoo, S.W.; Jung, D.; Min, J.-J.; Kim, H.; Lee, C. Biodegradable Contrast Agents for Photoacoustic Imaging. Appl. Sci. 2018, 8, 1567. [Google Scholar] [CrossRef] [Green Version]
- Park, B.; Lee, K.M.; Park, S.; Yun, M.; Choi, H.-J.; Kim, J.; Lee, C.; Kim, H.; Kim, C. Deep Tissue Photoacoustic Imaging of Nickel(II) Dithiolene-Containing Polymeric Nanoparticles in the Second near-Infrared Window. Theranostics 2020, 10, 2509–2521. [Google Scholar] [CrossRef]
- Lee, C.; Kwon, W.; Beack, S.; Lee, D.; Park, Y.; Kim, H.; Hahn, S.K.; Rhee, S.-W.; Kim, C. Biodegradable Nitrogen-Doped Carbon Nanodots for Non-Invasive Photoacoustic Imaging and Photothermal Therapy. Theranostics 2016, 6, 2196–2208. [Google Scholar] [CrossRef] [Green Version]
- Yao, J.; Wang, L.; Yang, J.-M.; Gao, L.S.; Maslov, K.I.; Wang, L.V.; Huang, C.-H.; Zou, J. Wide-Field Fast-Scanning Photoacoustic Microscopy Based on a Water-Immersible MEMS Scanning Mirror. J. Biomed. Opt. 2012, 17, 080505. [Google Scholar] [CrossRef]
- Kim, J.Y.; Lee, C.; Park, K.; Lim, G.; Kim, C. Fast Optical-Resolution Photoacoustic Microscopy Using a 2-Axis Water-Proofing MEMS Scanner. Sci. Rep. 2015, 5, 7932. [Google Scholar] [CrossRef]
- Kim, J.Y.; Lee, C.; Park, K.; Han, S.; Kim, C. High-Speed and High-SNR Photoacoustic Microscopy Based on a Galvanometer Mirror in Non-Conducting Liquid. Sci. Rep. 2016, 6, 34803. [Google Scholar] [CrossRef]
- Lee, C.; Lee, D.; Zhou, Q.; Kim, J.; Kim, C. Real-Time Near-Infrared Virtual Intraoperative Surgical Photoacoustic Microscopy. Photoacoustics 2015, 3, 100–106. [Google Scholar] [CrossRef] [Green Version]
- Shao, P.; Shi, W.; Chee, R.K.; Zemp, R.J. Mosaic Acquisition and Processing for Optical-Resolution Photoacoustic Microscopy. J. Biomed. Opt. 2012, 17, 080503. [Google Scholar] [CrossRef] [Green Version]
- Cho, S.; Baik, J.; Managuli, R.; Kim, C. 3D PHOVIS: 3D Photoacoustic Visualization Studio. Photoacoustics 2020, 18, 100168. [Google Scholar] [CrossRef]
- Zhao, H.; Chen, N.; Li, T.; Zhang, J.; Lin, R.; Gong, X.; Song, L.; Liu, Z.; Liu, C. Motion Correction in Optical Resolution Photoacoustic Microscopy. IEEE Trans. Med. Imaging 2019, 38, 2139–2150. [Google Scholar] [CrossRef] [PubMed]
- Tareen, S.A.K.; Saleem, Z. A Comparative Analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. In Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan, 3–4 March 2018. [Google Scholar] [CrossRef]
- Zhang, Z.; Lee, W.S. Deep Graphical Feature Learning for the Feature Matching Problem. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea, 27 October–2 November 2019; pp. 5087–5096. [Google Scholar] [CrossRef]
- Low, D.G. Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar] [CrossRef]
- Bay, H.; Tuytelaars, T.; Van Gool, L. SURF: Speeded Up Robust Features. In Computer Vision–ECCV 2006; Leonardis, A., Bischof, H., Pinz, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 404–417. [Google Scholar]
- Alcantarilla, P.F.; Nuevo, J.; Bartoli, A. Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces. In Proceedings of the British Machine Vision Conference, Bristol, UK, 9–13 September 2013; pp. 13.1–13.11. [Google Scholar] [CrossRef] [Green Version]
- Fischler, M.A.; Bolles, R.C. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. ACM 1981, 24, 381–395. [Google Scholar] [CrossRef]
- Lin, T.-Y.; Maire, M.; Belongie, S.; Bourdev, L.; Girshick, R.; Hays, J.; Perona, P.; Ramanan, D.; Zitnick, C.L.; Dollár, P. Microsoft COCO: Common Objects in Context. arXiv 2015, arXiv:14050312. [Google Scholar]
- Hernandez-Matas, C.; Zabulis, X.; Triantafyllou, A.; Anyfanti, P.; Douma, S.; Argyros, A.A. FIRE: Fundus Image Registration Dataset. Model. Artif. Intell. Ophthalmol. 2017, 1, 16–28. [Google Scholar] [CrossRef]
- Loshchilov, I.; Hutter, F. Decoupled Weight Decay Regularization. arXiv 2019, arXiv:171105101. [Google Scholar]
- Cielo, S.; Iapichino, L.; Baruffa, F. Speeding Simulation Analysis up with Yt and Intel Distribution for Python. arXiv 2019, arXiv:191007855. [Google Scholar]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Le, T.D.; Kwon, S.Y.; Lee, C. Performance Comparison of Feature Generation Algorithms for Mosaic Photoacoustic Microscopy. Photonics 2021, 8, 352. https://doi.org/10.3390/photonics8090352
Le TD, Kwon SY, Lee C. Performance Comparison of Feature Generation Algorithms for Mosaic Photoacoustic Microscopy. Photonics. 2021; 8(9):352. https://doi.org/10.3390/photonics8090352
Chicago/Turabian StyleLe, Thanh Dat, Seong Young Kwon, and Changho Lee. 2021. "Performance Comparison of Feature Generation Algorithms for Mosaic Photoacoustic Microscopy" Photonics 8, no. 9: 352. https://doi.org/10.3390/photonics8090352