Improving Imaging Quality of Real-time Fourier Single-pixel Imaging via Deep Learning
Abstract
:1. Introduction
2. Principles and Methods
2.1. Fourier Single-Pixel Imaging
2.2. Deep Learning Based FSPI
3. Results and Discussion
3.1. Simulations
3.2. Physical Experiments
- (1)
- Experiment 1: In the first experiment, the under-sampled images were acquired (through FSPI) from the imaging setup, and then the network was trained on those images for reconstruction.
- (2)
- Experiment 2: In the second experiment, the DL-FSPI model (DCAN block in Figure 10) trained on STL-10 dataset was applied directly onto the data from the imaging setup (under-sampled FSPI based images).
4. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Shapiro, J. Computational ghost imaging. Phys. Rev. A 2008, 78, 061802. [Google Scholar] [CrossRef]
- Zhang, Z.; Ma, X.; Zhong, J. Single-pixel imaging by means of Fourier spectrum acquisition. Nat. Commun. 2015, 6, 6225. [Google Scholar] [CrossRef] [PubMed]
- Ferri, F.; Magatti, D.; Lugiato, L.; Gatti, A. Differential ghost imaging. Phys. Rev. Lett. 2010, 104, 253603. [Google Scholar] [CrossRef] [PubMed]
- Sun, B.; Welsh, S.S.; Edgar, M.P.; Shapiro, J.H.; Padgett, M.J. Normalized ghost imaging. Opt. Express 2012, 20, 16892–16901. [Google Scholar] [CrossRef]
- Sun, B.; Edgar, M.P.; Bowman, R.; Vittert, L.E.; Welsh, S.; Bowman, A.; Padgett, M.J. 3D Computational Imaging with Single-Pixel Detectors. Science 2013, 340, 844–847. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Welsh, S.S.; Edgar, M.P.; Bowman, R.; Jonathan, P.; Sun, B.; Padgett, M.J. Fast full-color computational imaging with single-pixel detectors. Opt. Express 2013, 21, 23068–23074. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Z.; Wang, X.; Zheng, G.; Zhong, J. Hadamard single-pixel imaging versus Fourier single-pixel imaging. Opt. Express 2017, 25, 19619–19639. [Google Scholar] [CrossRef]
- Peng, J.; Yao, M.; Cheng, J.; Zhang, Z.; Li, S.; Zheng, G.; Zhong, J. Micro-tomography via single-pixel imaging. Opt. Express 2018, 26, 31094–31105. [Google Scholar] [CrossRef]
- He, Y.; Wang, G.; Dong, G.; Zhu, S.; Chen, H.; Zhang, A.; Xu, Z. Ghost imaging based on deep learning. Sci. Rep. 2018, 8, 6469. [Google Scholar] [CrossRef]
- Zhang, Z.; Wang, X.; Zheng, G.; Zhong, J. Fast Fourier single-pixel imaging via binary illumination. Sci. Rep. 2017, 7, 12029. [Google Scholar] [CrossRef]
- Huang, J.; Shi, D.; Yuan, K.; Hu, S.; Wang, Y. Computational-weighted Fourier single-pixel imaging via binary illumination. Opt. Express 2018, 26, 16547–16559. [Google Scholar] [CrossRef]
- Katkovnik, V.; Astola, J. Compressive sensing computational ghost imaging. J. Opt. Soc. Am. A 2012, 29, 1556–1567. [Google Scholar] [CrossRef]
- Katz, O.; Bromberg, Y.; Silberberg, Y. Compressive ghost imaging. Appl. Phys. Lett. 2009, 95, 131110. [Google Scholar] [CrossRef] [Green Version]
- Candes, E.; Wakin, M. An Introduction to Compressive Sampling. IEEE Signal Process. Mag. 2008, 25, 21–30. [Google Scholar] [CrossRef]
- Donoho, D.L. Compressed sensing. IEEE Trans. Inf. Theory 2006, 52, 1289–1306. [Google Scholar] [CrossRef]
- Cai, B.; Tao, D.; Xu, X.; Jia, K.; Qing, C. DehazeNet: An End-to-End System for Single Image Haze Removal. IEEE Trans. Image Process. 2016, 25, 5187–5198. [Google Scholar] [CrossRef] [Green Version]
- Satat, G.; Tancik, M.; Gupta, O.; Heshmat, B.; Raskar, R. Object classification through scattering media with deep learning on time resolved measurement. Opt. Express 2017, 25, 17466. [Google Scholar] [CrossRef]
- Caramazza, P.; Boccolini, A.; Buschek, D.; Hullin, M.; Higham, C.F.; Henderson, R.; Murray-Smith, R.; Faccio, D. Neural network identification of people hidden from view with a single-pixel, single-photon detector. Sci. Rep. 2018, 8, 11945. [Google Scholar] [CrossRef]
- Sinha, A.; Lee, J.; Li, S.; Barbastathis, G. Lensless computational imaging through deep learning. Optica 2017, 4, 1117–1125. [Google Scholar] [CrossRef] [Green Version]
- Higham, C.F.; Murray-Smith, R.; Padgett, M.J.; Edgar, M.P. Deep learning for real-time single-pixel video. Sci. Rep. 2018, 8, 2369. [Google Scholar] [CrossRef]
- Vincent, P.; LaRochelle, H.; Bengio, Y.; Manzagol, P.-A. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 5–9 July 2008; pp. 1096–1103. [Google Scholar]
- Xu, Z.-H.; Chen, W.; Penuelas, J.; Padgett, M.; Sun, M.-J. 1000 fps computational ghost imaging using LED-based structured illumination. Opt. Express 2018, 26, 2427–2434. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Glorot, X.; Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy, 13–15 May 2010. [Google Scholar]
- Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 448–456. [Google Scholar]
- Kingma, D.; Ba, J. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Coates, A.; Lee, H.; Ng, A.Y. An analysis of single layer networks in unsupervised feature learning. In Proceedings of the Artificial Intelligence and Statistics Conference, Fort Lauderdale, FL, USA, 11–13 April 2011. [Google Scholar]
- Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 3, 600–612. [Google Scholar] [CrossRef]
Method | Acquisition Time (operating DMD at ~22 kHz) | Reconstruction Time (IFT or IFT+DL) | Imaging Time | Frames per Second (fps) |
---|---|---|---|---|
FSPI 25% | 157 ms | 9 ms | 166 ms | 6 |
DL-FSPI-10 | 63 ms | 21 ms | 84 ms | 12 |
DL-FSPI-8 | 50 ms | 21 ms | 71 ms | 14 |
DL-FSPI-6 | 38 ms | 21 ms | 59 ms | 16 |
DL-FSPI-5 | 31 ms | 21 ms | 52 ms | 18 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Rizvi, S.; Cao, J.; Zhang, K.; Hao, Q. Improving Imaging Quality of Real-time Fourier Single-pixel Imaging via Deep Learning. Sensors 2019, 19, 4190. https://doi.org/10.3390/s19194190
Rizvi S, Cao J, Zhang K, Hao Q. Improving Imaging Quality of Real-time Fourier Single-pixel Imaging via Deep Learning. Sensors. 2019; 19(19):4190. https://doi.org/10.3390/s19194190
Chicago/Turabian StyleRizvi, Saad, Jie Cao, Kaiyu Zhang, and Qun Hao. 2019. "Improving Imaging Quality of Real-time Fourier Single-pixel Imaging via Deep Learning" Sensors 19, no. 19: 4190. https://doi.org/10.3390/s19194190