Multicolor Holographic Display of 3D Scenes Using Referenceless Phase Holography (RELPH)

Abstract

In this paper, we present a multicolor display via referenceless phase holography (RELPH). RELPH permits the display of full optical wave fields (amplitude and phase) using two liquid crystal phase-only spatial light modulators in a Michelson-interferometer-based arrangement. Complex wave fields corresponding to arbitrary real or artificial 3D scenes are decomposed into two mutually coherent wave fields of constant amplitude whose phase distributions are modulated onto the wave fields reflected by the respective light modulators. Here, we present the realization of that concept in two different ways: firstly, via temporal multiplexing using a single setup, switching between wavelengths for temporal integration of the respective wavefields; secondly, using spatial multiplexing of different wavelengths with multiple Michelson-based setups; and finally, we present an approach to magnify the 3D scenes displayed by light modulators with limited space–bandwidth product for a comfortable viewing experience.

1. Introduction

Holography still represents—at least in principle—the gold standard of three-dimensional display [1,2,3]. In contrast to volumetric displays [4,5] as well as stereoscopic or light field technologies [6,7], it allows to satisfy all depth cues of the human perception, including full parallax, a large depth of field, and a wide angle of view [8]. While holography is well-established in the case of static scenes, it still remains a technological challenge to realize dynamic holographic displays. One of the primary problems is the low space–bandwidth product (SBP) of currently available dynamic spatial light modulators (SLM) [9]. To overcome the problem of low SBP, approaches including temporal multiplexing [10,11,12,13], spatial multiplexing [14], a combination of static and dynamic light modulators [15], rewritable photographic media [16] or eye-tracking [17,18] have been investigated.

Regardless of limitations due to the SBP of available SLMs, there have been considerable efforts towards dynamical holographic multicolor displays, superpositioning multiple colors through frequency domain multiplexing [19,20], temporal multiplexing [21,22] or spatial multiplexing [23]. To improve the image quality of these multicolor holographic displays, multiple techniques have been investigated, including deep neural networks [24], camera-in-the-loop techniques [25], as well as the combination of holography with light field-based approaches [26].

Recently, referenceless phase holography (RELPH) has been introduced [27,28], which aims at the efficient use of the available SBP of current SLMs. Instead of recreating the holographically recorded interference pattern, as obtained from the superposition of an object wave with a reference wave, it makes use of multiple phase-only SLMs to generate a desired complex amplitude at a given plane in space. The great advantage is that no zero-order and no pseudoscopic images arise, leading to an optimum use of the available SBP.

In this paper, we present the multicolor holographic display of 3D scenes by the RELPH method. To reconstruct the wave fields, we use a Michelson-interferometer-based setup with two SLMs modulating the respective phase distributions. We propose two different methods for multicolor display, firstly by temporal multiplexing of three different wavelengths, displayed in temporal sequence. The second given color multiplexing method is based on superpositioning the reconstructed holographic wave fields of different wavelengths spatially. For this, two separate Michelson-based setups are used, where the simultaneous spatial superposition of the respective reconstructed wave fields results in a multicolor impression.

2. Mathematical Description and Experimental Setup

2.1. Mathematical Description of RELPH

The holographical display of 3D scenes is based on the reconstruction of complex 2D diffraction fields in a given hologram plane $(x,y)$ [29]. The diffraction field $E(x,y)$, with the intensity distribution $I(x,y)$ and the phase distribution $\varphi (x,y)$, can, without loss of generality, be normalized for the intensity distribution to have values only within the interval $[0,4]$ [27]. Any such wave field E can be decomposed into two mutually coherent wave fields, $E_1(x,y)$ and $E_2(x,y)$, consisting of pure phase distributions, ${\varphi }_1(x,y)$ and ${\varphi }_2(x,y)$, with a constant amplitude,

$$\begin{array}{cc}\hfill E(x,y)& =\sqrt{I(x,y)}·exp\left[i\varphi (x,y)\right]\hfill \\ \hfill & =E_1(x,y)+E_2(x,y)\hfill \\ \hfill & =exp\left[i{\varphi }_1(x,y)\right]+exp\left[i{\varphi }_2(x,y)\right],\hfill \end{array}$$

(1)

where the two phase distributions are given by

$$\begin{array}{c}\hfill {\varphi }_1(x,y)=\varphi (x,y)+arctan\left(\sqrt{(4-I(x,y\left)\right)/I(x,y)}\right)\end{array}$$

(2)

$$\begin{array}{c}\hfill {\varphi }_2(x,y)=\varphi (x,y)-arctan\left(\sqrt{(4-I(x,y\left)\right)/I(x,y)}\right),\end{array}$$

(3)

as shown in [27].

To find the wave field $E(x,y)$ necessary to display a 3D object holographically, the object is sliced into 2D planes. Each plane $(\epsilon ,\eta ,z)$ contains the 3D objects’ points at the z-coordinate of the respective plane, with $E^{\prime }(\epsilon ,\eta ,z)$ being the wave field emitted from these object points. The wave field distribution $E(x,y)$ in the SLM plane ($z=0$) is calculated by propagating each individual wave field $E^{\prime }(\epsilon ,\eta ,z)$ into that plane by the angular spectrum method [30]. The individually propagated wave fields from each z-coordinate are summed up for coherent superposition in the SLM plane,

$$\begin{array}{cc}\hfill E(x,y)& =\sum _{z\in g}{\mathcal{F}}^{-1}\left\{\mathcal{F}\left\{E^{\prime }(\epsilon ,\eta ,z)\right\}·H(f_x,f_y)\right\},\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill H(f_x,f_y)& =\left\{\begin{array}{cc}{e}^{iz2\pi /\lambda \sqrt{1-{\left(\lambda f_x\right)}^2-{\left(\lambda f_y\right)}^2}},\hfill & \mathrm{if}\sqrt{f_x^2+f_y^2}<1/\lambda \hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(5)

where the sum is written over the z-coordinates of the individual slices along a linear array g, with the transfer function H. $\mathcal{F}$ denotes the two-dimensional Fourier transform, with the spatial frequencies in Fourier space $f_x=\frac{1}{N\Delta \epsilon },f_y=\frac{1}{M\Delta \eta }$, as defined by the discrete Fast Fourier Transform. Here, $N,M$ are the number of pixels and $\Delta \epsilon ,\Delta \eta$ are the pixel pitches of the SLMs in vertical and horizontal directions, and $\lambda$ is the wavelength. The intensity and phase distributions $I(x,y)$ and $\varphi (x,y)$ are then calculated from this propagated and superpositioned wave field $E(x,y)$ to find the phase distributions ${\varphi }_1(x,y),{\varphi }_2(x,y)$ for each SLM according to Equations (2) and (3).

2.2. Setup of RELPH

The 3D scenes are displayed using two identical phase-only liquid crystal spatial light modulators (SLM) in a Michelson-interferometer-based setup shown in Figure 1a. The superposition of the two wave fields given by Equation (1) is realized by placing one SLM in each interferometer arm of a Michelson-interferometer. Due to the SLMs birefringence, a polarizer is placed in front of the beam-splitter, corresponding to the slow axis of the SLMs, as well as an additional (optional) analyzer after the beam-splitter. For a multicolor display, three lasers with different wavelengths corresponding to the three fundamental colors, red, green, and blue are used. The light coming from the lasers is parallelized and expanded by three lenses to produce plane waves with the diameter of the SLMs. A 3D display via the RELPH method using such a setup has previously been shown to display the 3D cues’ depth of field, occlusion, and parallax [28].

For spatial multiplexing of two wave fields with different colors, two Michelson-based setups are coupled via an additional beam-splitter, as shown in Figure 1b. Generally, the optical path between the SLM planes of the respective RELPH setups and the additional beam-splitter will differ. This has to be taken into account when defining the imaging planes for the respective setups. Three-dimensional scenes can be displayed as real images between the SLM plane and the viewer or as virtual images behind the SLM plane. For the experimental verification of the displayed 3D scenes, the intensity of the reconstructed wave fields is recorded by a CCD camera. To record virtual 3D scenes, a simple plano-convex lens is used to image the virtual scene onto the CCD camera.

Since the SLMs are arranged in a Michelson-based arrangement, there will be interference patterns due to out-of-plane rotations, as well as mutual deformation of the SLMs. Additionally, proper lateral pixel-to-pixel alignment of the two SLMs is crucial for the unperturbed display of 3D scenes, since non-perfect alignment causes residual interferences. In [28], a procedure for proper alignment of the SLMs is described, containing the display of auxiliary grid structures by the SLMs.

When addressing the SLMs with a constant phase of zero, they are assumed to act as plane mirrors. However, low-spatial-frequency manufacturing deformations in the µm range add additional phase terms ${\phi }_1(x,y),{\phi }_2(x,y)$, where the mutual phase difference $\Delta \phi (x,y)={\phi }_1(x,y)-{\phi }_2(x,y)$ generates spurious interference patterns. To compensate this phase difference, $\Delta \phi$ phase-shifting can be applied by recording several interferograms while addressing the SLMs with constant phases of defined phase differences. Thus, to compensate the mutual deformation of the SLMs, for arbitrary RELPH-generated holograms, an adjusted phase ${\varphi }_1^{\prime }={\varphi }_1-\Delta \phi$ has to be displayed.

2.3. Magnification of RELPH-Generated 3D Scenes

The number of pixels on a SLM defines the SBP. There will always be a trade-off between the lateral size of modulated wave fields and the maximum diffraction or opening angle of displayed 3D scenes. For a comfortable viewing experience in different applications, the way that the area and angle shall be distributed may vary decidedly. Thus, it may be necessary to reshape this trade-off for a given SBP. This can be achieved by adding a magnifying imaging system to the setup.

A two-lens setup is utilized for magnification of displayed 3D scenes, where the second lens is the main aperture of the magnified wave field. As depicted in the schematic sketch of the magnification setup given in Figure 2, a holographically reconstructed 3D scene, which is premagnified by lens L${}_1$, gets subsequently imaged to a magnified virtual or real image by lens L${}_2$. Additionally, the second lens redirects diverging light back towards the optical axis to enable observation of the 3D scene by optics with small aperture (e.g., the human pupil). According to geometric optics, a magnified object to be displayed in a distance $z_{\mathrm{mag}}$ from L${}_2$ has to be displayed by the RELPH setup in a distance

$$\begin{array}{c}\hfill z_0=\frac{f_1^2}{d-f_1-\frac{f_2·z_{\mathrm{mag}}}{z_{\mathrm{mag}}-f_2}}+f_1\end{array}$$

(6)

from L₁. The magnification of the object

$$\begin{array}{c}\hfill m\left(z_0\right)=\frac{f_1·f_2}{z_0·(f_2-d)-f_1·(f_2-z_0-d)}\end{array}$$

(7)

is dependent on its position. Thus, when displaying magnified scenes in the distance $z_{\mathrm{mag}}$ from L${}_2$, each 2D slice of the 3D scenes has to be resized and propagated according to Equations (6) and (7). When displaying a 3D scene consisting of multiple depth planes, the depth plane with the smallest magnification $m\left(z_0\right)$ should, at maximum, fill the whole SLM, and subsequent planes are scaled to counteract the increase in magnification.

3. Experimental Results

We demonstrate two different approaches to display multicolor 3D scenes: (i) firstly by using temporal multiplexing of three wave fields, and (ii) by a multicolor display of RELPH-generated holograms superpositioning two different wavelengths spatially.

3.1. Temporal Multiplexing

Using the setup shown in Figure 1a, we obtain temporal multiplexing of three wave fields with wavelengths 633 nm, 532 nm, and 488 nm by changing the wavelength in temporal sequence. Since the SLMs are capable of switching in video rate (60 Hz), switching between the wavelength electronically at the same rate allows a live multicolor hologram. For simplicity, within this demonstration we manually change the wavelength between recording the reconstruction of each individual color channel. Both the propagation of the wave field described in Equation (5), as well as the modulation by the SLM are wavelength-dependent. Thus, even for 3D scenes containing identical intensity information in all three color channels, modulation by the SLM is different for each color channel.

The SLM modulates the phase of a wave field by varying the refractive index, and consequently, the optical path length experienced by the wave field. To adjust for the wavelength dependence of the SLMs, we assume constant dispersion. This results in a simple multiplicative amplitude factor $a=\lambda /{\lambda }_0$ scaling the RELPH-generated phases $a·{\varphi }_1,a·{\varphi }_2$, where ${\lambda }_0$ is the wavelength that the SLM is initially calibrated for. The wavelength dependence of the propagation is inherently addressed when generating the RELPH-phases from Equation (5).

To demonstrate the multicolor display of RELPH-holograms via temporal multiplexing, we present an obliquely oriented wire frame cube of constant intensity along its edges across all three color channels. Its front and rear faces are placed at distances of 150 mm and 130 mm in front of the SLM plane, respectively. Within the wire frame cube, the letters BIAS are displayed in different colors, while being placed in descending distances in the same distance range as above. For each color channel, the 3D scene was sliced into 51 parallel equidistant slices which were propagated and superpositioned into the SLM plane, where the resulting wave field was decomposed into two phase-only wave fields of constant amplitude $E_1$ and $E_2$. As an example, the phase distributions of each color channel for one of the SLMs are displayed in Figure 3.

The wave field was captured by a CCD camera with the lens positioned to focus on either the front or rear faces of the wire frame cube. In Figure 4, the captured 3D scene is shown, where the three images corresponding to the different color channels were recorded in temporal sequence. The image shown in Figure 4a is focused on the front of the 3D scene, with the three color channels combining for a sharp and white front face of the cube, the letter B also being in focus, and the other letters as well as the rear face being increasingly out of focus, as expected. In turn, in Figure 4b, the focus is on the rear side, and the letter S displays the 3D cue depth of field. A constant dim background is present due to the light reflected from inactive areas of the SLMs corresponding to their fill factor [31].

3.2. Spatial Multiplexing

In order to demonstrate a multicolor display of RELPH-generated holograms, we use the wavelengths of 633 nm and 532 nm to demonstrate the concept of simultaneous spatial multiplexing of different colors according to the setup shown in Figure 1b. To define a mutual imaging plane of the separate RELPH setups, the optical path difference that generally exists between the SLM planes of the respective RELPH-setups and the beam-splitter combining the two setups has to be determined. This was done by displaying auxiliary grid lines propagated into depth planes of varying distances from the respective SLM planes until the grid lines from both setups were propagated into the same focal plane. For the subsequent display of 3D scenes, this optical path difference was compensated when calculating the diffraction patterns for the respective setups. The addition of a third color for RGB display would be analogous to this method.

The extent of the displayed 3D scenes is mainly limited by the size of SLM of 15.36 × 8.64 mm, as well as the maximum diffraction angle of the SLM being around $2^{\circ }$ when using light in the visible range due to the SLMs’ pixel pitch of 8 µm. With the number of pixels of the SLM ($1920\times 1080$) defining a fixed SBP and thus the trade-off between the angle and area of a modulated wave field, we present a simple way to reshape this relationship for a more comfortable viewing experience by magnifying the 3D scene using the two-lens setup shown in Figure 2. The focal length of the lenses L${}_1$ and L${}_2$ are $f_1=30$ mm and $f_2=250$ mm, and the diameter of L${}_2$ is 150 mm, effectively defining the aperture of the magnified scene. The lenses are placed in a distance of $d=340$ mm with respect to one another.

To calculate the diffraction patterns, the magnifying setup had to be taken into account, with each 2D slice of the 3D scene being resized and propagated according to Equations (6) and (7). The magnified cube was displayed as a real image laying between 15 mm and 50 mm behind L${}_2$, with the edges being 35 mm long, and the front and rear sides being 35 mm apart. This corresponds to an unmagnified cube being generated by the SLMs with edges 3.6 mm long between planes laying 32.8 mm and 32.4 mm in front of L${}_1$.

The magnified 3D scene was captured using a color camera with a lens focusing on the front side of the wire frame cube. While in the unmagnified case, a depth impression was apparent when focusing on either faces of the cube, and in Figure 5 no such depth impression is present, as both front and rear sides appear to be equally in focus at the same time. When magnifying 3D scenes, the limited diffraction angle of the SLMs restricts the depth of field, leading to a limited depth perception since the etendue or geometrical extent is conserved. Thus, for magnified wave fields, objects in different depths appear to lie in similar focal planes. To avert this effect, SLMs with a higher space–bandwidth product, containing more and smaller pixels are necessary.

Compared to the 3D scenes generated by temporal multiplexing shown in Figure 4, the magnified reconstruction via spatial multiplexing suffers from further degradation regarding the image quality. The chromatic aberration, which is more pronounced due to the magnifying setup, was at least partially compensated by rescaling of the RELPH-generated holograms and adjusting the propagation distances. Furthermore, the employment of two distinct RELPH setups results in possible alignment errors, and different focusabilities of the RELPH reconstructions, due to different optical paths of the two setups, as well other aberrations, render an additional mutual calibration of the two RELPH setups necessary.

4. Discussion

Referenceless phase holography (RELPH) provides a novel approach for the holographical multicolor display of 3D scenes. With holography of static scenes being well-established, the dynamic holographic display of 3D scenes remains a challenge due to limited space–bandwidth products (SBP) of light modulators. Using phase-only spatial light modulators in Michelson-interferometer-based setups, RELPH reconstructs arbitrary full wave fields (amplitude and phase) while utilizing the SBP of commercially available spatial light modulators in an optimal way. In contrast to classical holography, no zeroth-order diffraction or pseudoscopic images are present for RELPH-generated holograms.

We have presented here two different approaches for the multicolor display of 3D scenes, namely temporal and spatial multiplexing of different colors, superpositioning multiple wave fields in a temporal sequence or by using two separate Michelson-based setups using an additional beam-splitter for superpositioning, demonstrating that the recently introduced referenceless phase holography is a possible candidate for applications such as holographic near-eye 3D displays, medical diagnostics, or industrial design and prototyping. In the case of spatial multiplexing, we presently find a further degradation of the image quality which requires an additional mutual calibration of the RELPH setups for each color for further development. Improvement in the image quality might also be achieved using convolutional neural networks [24] or camera-in-the-loop technology [25].

Currently, a main limitation of holographic 3D displays is the limited SBP of available SLMs. However, for certain potential applications of a RELPH display, it may be suitable to sacrifice available viewing angles for an increased lateral size of the 3D images. For this purpose, we demonstrated a straightforward way to magnify the 3D scene, increasing the overall lateral size to around 6.8 cm but decreasing the viewing angle. However, this procedure resulted in a loss of depth impression, demonstrating the need for SLMs with higher SBP for demanding commercial applications.