Holographic 3D Display Using Depth Maps Generated by 2D-to-3D Rendering Approach
Abstract
:1. Introduction
2. Generation of Depth Maps
2.1. Distant View Images
2.2. Perspective View Images
2.3. Close-Up Images
3. Calculation and Reconstructions of CGHs
3.1. Calculation of CGHs
3.2. Reconstruction of CGHs
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Kim, M.K. Full color natural light holographic camera. Opt. Express 2013, 21, 9636–9642. [Google Scholar] [CrossRef] [PubMed]
- Lee, K.R.; Park, Y.K. Exploiting the speckle-correlation scattering matrix for a compact reference-free holographic image sensor. Nat. Commun. 2016, 7, 13359. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Antipa, N.; Kuo, G.; Heckel, R.; Mildenhall, B.; Bostan, E.; Ng, R.; Waller, L. DiffuserCam: Lensless single-exposure 3D imaging. Optica 2018, 5, 1–9. [Google Scholar] [CrossRef]
- Bradley, M.; Sabatier, J.M. Applications of Fresnel-Kirchhoff diffraction theory in the analysis of human-motion Doppler sonar grams. J. Acoust. Soc. Am. 2010, 128, EL248. [Google Scholar] [CrossRef] [PubMed]
- Zhang, W.; Hoorfar, A. Three-dimensional real-time through-the-wall radar imaging with diffraction tomographic algorithm. IEEE Trans. Geosci. Remote Sens. 2013, 51, 4155–4163. [Google Scholar] [CrossRef]
- Lee, M.K. Wide area quantitative phase microscopy by spatial phase scanning digital holography. Opt. Lett. 2020, 45, 784–786. [Google Scholar]
- Wu, L.; Zhang, Z. Domain multiplexed computer-generated holography by embedded wavevector filtering algorithm. PhotoniX 2021, 2, 1. [Google Scholar] [CrossRef]
- Wang, Z.; Zhang, X.; Lv, G.; Feng, Q.; Ming, H.; Wang, A. Hybrid holographic Maxwellian near-eye display based on spherical wave and plane wave reconstruction for augmented reality display. Opt. Express 2021, 29, 4927–4935. [Google Scholar] [CrossRef]
- Wang, Z.; Zhang, X.; Tu, K.; Lv, G.; Feng, Q.; Wang, A.; Ming, H. Lensless full-color holographic Maxwellian near-eye display with a horizontal eyebox expansion. Opt. Lett. 2021, 46, 4112–4115. [Google Scholar] [CrossRef]
- Wang, D.; Zheng, Y.-W.; Li, N.-N.; Wang, Q.-H. Holographic display system to suppress speckle noise based on beam shaping. Photonics 2021, 8, 204. [Google Scholar] [CrossRef]
- He, Z.; Sui, X.; Zhang, H.; Jin, G.; Cao, L. Frequency-based optimized random phase for computer-generated holographic display. Appl. Opt. 2021, 60, A145–A154. [Google Scholar] [CrossRef]
- Wang, D.; Liu, C.; Shen, C.; Wang, Q.-H. Holographic capture and projection system of real object based on tunable zoom lens. PhotoniX 2020, 1, 6. [Google Scholar] [CrossRef] [Green Version]
- Zhao, Y.; Cao, L.; Zhang, H.; Tan, W.; Wu, S.; Wang, Z.; Yang, Q.; Jin, G. Time-division multiplexing holographic display using angular-spectrum layer-oriented method. Chin. Opt. Lett. 2016, 14, 010005. [Google Scholar] [CrossRef]
- Keil, J.; Edler, D.; Dickmann, F. Preparing the Hololens for user studies: An augmented reality interface for the spatial adjustment of holographic objects in 3D indoor environments. J. Cartogr. Geogr. Inf. 2019, 69, 205–215. [Google Scholar] [CrossRef] [Green Version]
- Wyss, C.; Bührer, W.; Furrer, F.; Degonda, A.; Hiss, J.A. Innovative teacher education with the augmented reality device Microsoft Hololens—results of an exploratory study and pedagogical considerations. Multimodal Technol. Interact. 2021, 5, 45. [Google Scholar] [CrossRef]
- Keil, J.; Korte, A.; Ratmer, A.; Edler, D.; Dickmann, F. Augmented reality (AR) and spatial cognition: Effects of holographic grids on distance estimation and location memory in a 3D indoor scenario. J. Photogramm. Remote Sens. Geoinf. Sci. 2020, 88, 165–172. [Google Scholar] [CrossRef]
- Moro, C.; Phelps, C.; Redmond, P.; Stromberga, Z. HoloLens and mobile augmented reality in medical and health science education: A randomised controlled trial. Br. J. Educ. Technol. 2020, 52, 680–694. [Google Scholar] [CrossRef]
- Mishina, T.; Okui, M.; Okano, F. Calculation of holograms from elemental images captured by integral photography. Appl. Opt. 2006, 45, 4026–4036. [Google Scholar] [CrossRef]
- Yanagihara, H.; Kakue, T.; Yamamoto, Y.; Shimobaba, T.; Ito, T. Real-time three-dimensional video reconstruction of real scenes with deep depth using electro-holographic display system. Opt. Express 2019, 27, 15662–15678. [Google Scholar] [CrossRef]
- Yamaguchi, M. Light-field and holographic three-dimensional displays. J. Opt. Soc. Am. A 2016, 33, 2348–2364. [Google Scholar] [CrossRef]
- Igarashi, S.; Nakamura, T.; Matsushima, K.; Yamaguchi, M. Efficient tiled calculation of over-10-gigapixel holograms using ray-wavefront conversion. Opt. Express 2018, 26, 10773–10786. [Google Scholar] [CrossRef]
- Tsai, S.-F.; Cheng, C.-C.; Li, C.-T.; Chen, L.-G. A real-time 1080p 2D-to-3D video conversion system. IEEE Trans. Consum. Electron. 2011, 57, 915–922. [Google Scholar] [CrossRef]
- Cheng, C.-C.; Li, C.-T.; Chen, L.-G. A novel 2D-to-3D conversion system using edge information. IEEE Trans. Consum. Electron. 2010, 56, 1739–1745. [Google Scholar] [CrossRef]
- Lai, Y.-K.; Lai, Y.-F.; Chen, Y.-C. An effective hybrid depth-generation algorithm for 2D-to-3D conversion in 3D displays. J. Disp. Technol. 2013, 9, 154–161. [Google Scholar] [CrossRef]
- Zhang, Z.; Yin, S.; Liu, L.; Wei, S. A real-time time-consistent 2D-to-3D video conversion system using color histogram. IEEE Trans. Consum. Electron. 2015, 61, 524–530. [Google Scholar] [CrossRef]
- Gil, J.; Kim, M. Motion depth generation using MHI for 2D-to-3D video conversion. Electron. Lett. 2017, 53, 1520–1522. [Google Scholar] [CrossRef]
- Tsai, T.-H.; Fan, C.-S.; Huang, C.-C. Semi-automatic depth map extraction method for stereo video conversion. In Proceedings of the 6th International Conference on Genetic and Evolutionary Computing (ICGEC 2012), Kitakyushu, Japan, 25–28 August 2012; pp. 340–343. [Google Scholar]
- Boleček, L.; Říčný, V. The estimation of a depth map using spatial continuity and edges. In Proceedings of the 36th International Conference on Telecommunications and Signal Processing (TSP 2013), Rome, Italy, 2–4 July 2013; pp. 890–894. [Google Scholar]
- Yang, Y.; Hu, X.; Wu, N.; Wang, P.; Xu, D.; Rong, S. A depth map generation algorithm based on saliency detection for 2D to 3D conversion. 3D Res. 2017, 8, 29. [Google Scholar] [CrossRef]
- Brox, T.; Bruhn, A.; Papenberg, N.; Weickert, J. High accuracy optical flow estimation based on a theory for warping. In Proceedings of the 8th European Conference on Computer Vision (ECCV 2004), Prague, Czech Republic, 11–14 May 2004; pp. 25–36. [Google Scholar]
- Laina, I.; Rupprecht, C.; Belagiannis, V.; Tombari, F.; Navab, N. Deeper depth prediction with fully convolutional residual networks. In Proceedings of the 4th International Conference on 3D Vision (3DV 2016), Stanford, CA, USA, 25–28 October 2016; pp. 239–248. [Google Scholar]
- Sun, D.; Yang, X.; Liu, M.-Y.; Kautz, J. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In Proceedings of the 2018 Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA, 18–23 June 2018; pp. 8934–8943. [Google Scholar]
- Song, M.; Kim, W. Depth estimation from a single image using guided deep network. IEEE Access 2019, 7, 142595–142606. [Google Scholar] [CrossRef]
- Ranftl, R.; Lasinger, K.; Hafner, D.; Schindler, K.; Koltun, V. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE Trans. Pattern Anal. Mach. Intell. 2020. early access. [Google Scholar] [CrossRef]
- Perez, F.; Koch, C. Toward color image segmentation in analog VLSI: Algorithm and hardware. Int. J. Comput. Vis. 1994, 12, 17–42. [Google Scholar] [CrossRef] [Green Version]
- Huang, Y.-S.; Cheng, F.-H.; Liang, Y.-H. Creating depth map from 2D scene classification. In Proceedings of the 2008 3rd International Conference on Innovative Computing Information and Control, Dalian, China, 18–20 June 2008; p. 69. [Google Scholar]
- Cheng, C.-C.; Li, C.-T.; Chen, L.-G. An ultra-low-cost 2-D/3-D video-conversion system. SID Symp. Dig. Tech. Pap. 2010, 41, 766–769. [Google Scholar] [CrossRef]
- Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 1986, 8, 679–698. [Google Scholar] [CrossRef]
- Ballard, D.H. Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognit. 1981, 13, 111–122. [Google Scholar] [CrossRef] [Green Version]
- Battiato, S.; Curti, S.; La Cascia, M.; Tortora, M.; Scordato, E. Depth map generation by image classification. Proc. SPIE 2004, 5302, 95–104. [Google Scholar]
- Yin, S.; Dong, H.; Jiang, G.; Liu, L.; Wei, S. A novel 2D-to-3D video conversion method using time-coherent depth maps. Sensors 2015, 15, 15246–15264. [Google Scholar] [CrossRef] [Green Version]
- Zhao, Y.; Cao, L.; Zhang, H.; Kong, D.; Jin, G. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method. Opt. Express 2015, 23, 25440–25449. [Google Scholar] [CrossRef]
- Wu, J.; Liu, K.; Sui, X.; Cao, L. High-speed computer-generated holography using an autoencoder-based deep neural network. Opt. Lett. 2021, 46, 2908–2911. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
He, Z.; Sui, X.; Cao, L. Holographic 3D Display Using Depth Maps Generated by 2D-to-3D Rendering Approach. Appl. Sci. 2021, 11, 9889. https://doi.org/10.3390/app11219889
He Z, Sui X, Cao L. Holographic 3D Display Using Depth Maps Generated by 2D-to-3D Rendering Approach. Applied Sciences. 2021; 11(21):9889. https://doi.org/10.3390/app11219889
Chicago/Turabian StyleHe, Zehao, Xiaomeng Sui, and Liangcai Cao. 2021. "Holographic 3D Display Using Depth Maps Generated by 2D-to-3D Rendering Approach" Applied Sciences 11, no. 21: 9889. https://doi.org/10.3390/app11219889