Near-Eye Holographic 3D Display and Advanced Amplitude-Modulating Encoding Scheme for Extended Reality

Abstract

Electronic holographic displays can reconstruct the optical wavefront of object light, exhibiting the most realistic three-dimensional (3D) images, in contrast to conventional stereoscopic displays. In this paper, we propose a novel, near-eye holographic 3D display (NEHD) applicable to AR/MR/holographic devices and experimentally demonstrate the proposed module’s performance with 360° full-viewed holographic 3D movie at 30 fps. To realize high-quality of reconstructed holographic 3D (H3D) images, we also propose an advanced amplitude-modulating (AM) encoding scheme suited for the proposed amplitude-modulating NEHD. We experimentally verify that the new hologram-encoding approach can improve the image quality of H3D reconstructions through quantitative statistical analyses, by using evaluation methods for H3D images that are suggested in the paper. Two holograms at different viewing directions of the same 3D scene are designed to be displayed onto the proposed NEHD prototype for two eyes of an observer, respectively. The presented techniques for the proposed NEHD enable the observer to experience the depth cue, a realistic accommodation effect, and high-quality H3D movies at each eye.

1. Introduction

Electronic holographic display (EHD) [1,2,3] is a light interference-based 3D technique that can provide non-glasses 3D scenes because it can reconstruct the natural wavefront of light emitted from objects. A hologram, including computer-generated hologram (CGHs), is an interference fringe pattern in a semi-transparent medium that is generated using the interference between the object light and the reference light [3,4]. In contrast, a stereoscopic 3D display technique is based on multiple two-dimensional images, resulting in accommodation–convergence conflict [2,5]. In general, the viewing angle depends on the pixel pitch of the spatial light modulator (SLM) used as a holographic 3D display [4,5]. To reconstruct a holographic 3D (H3D) image with a moderate wide-viewing angle, current technologies of holographic 3D displays cannot satisfy demand, the state-of-the-art for SLM of which the pixel pitch goes down into about $1\mathsf{\mu }\mathrm{m}$ or smaller [2,3,4]. Alternatively, either temporally or spatially multiplexed SLMs have been applied to increase this viewing angle in utilizing these conventional displays as well as the scalability of 3D image sizes [6,7,8,9], but these large-sized, complicated EHDs require further great advances in display technology towards a single integrated hardware device. In addition, a single conventional SLM cannot easily modulate both the amplitude and phase of an object’s light wave [8,9]. Recently, although a proposed, complex-modulating SLM including eye-tracking, referred to as a complex modulator, achieved conceptual improvement, it suffers from mechanically unsterilized alignment among micro-pixels between two stacked SLMs and also requires bulky optical components [10]. In some cases, an optical filtering system must be applied to remove the unwanted diffraction orders as well as background noise, resulting in enlarged display systems [6,9,10,11,12]. Alternatively, complex-to-phase or complex-to-amplitude encoding methods can be applied to convert the complex-number hologram into a phase-only modulator or an amplitude-only modulator [10,13,14,15,16,17]. With these existing approaches, the image quality of holographic 3D displays highly depends on the selected encoding types, and, therefore, alternative encoding methods should be further studied.

Recently, the near-eye displays for extended reality (XR) including augmented reality (AR)/virtual reality (VR) and mixed reality (MR) are providing potential to bring users into new worlds where they can experience instant access to stereoscopic and multi-viewed visual information [17,18,19]. As for EHD devices, the key requirements, such as enlarging the viewing angle of compact holographic device and representing the complex-field’s hologram in a 3D scene with high quality, can be easily overcome by merging a near-eye-typed display device and digitally encoded holographic content [17,18,19]. We began to pursue the very hybrid approach in the amplitude-modulating SLM to be embodied by using a novel near-eyed holographic display and an advanced encoding method that we present in the study. In this paper, we propose an advanced efficient encoding scheme, which we refer to as modified amplitude-only encoding (MAO), and from which a pair of encoded holograms per viewpoint are prepared through the image data processing for holographic 3D content, as depicted in Figure 1a. In addition, we demonstrate a prototype of the near-eye holographic 3D display (NEHD) suited for prepared 360° holographic content, as shown in Figure 1b. The module can split and upload a pair of MAO-encoded holograms simultaneously, and then optically reconstruct two H3D images which can be observed by the left and right eyes of an observer, respectively. The proposed dual-view NEHD architecture and MAO encoding scheme are aimed at compact and lightweight applications, thereby allowing direct integration into existing wearable mobile devices towards realizing a holographic 3D display for the user to experience immersive, realistic, and high-quality H3D movies.

2. Methods

2.1. Near-Eye Holographic 3D Display for Dual-View CGH Content

For the demonstrational environment of the proposed near-eye holographic 3D display (NEHD), we used the amplitude-modulating SLM module that consists of two micro-display panels (MDPs). Here, each panel (Model: RDP551F) is a kind of LCoS (Liquid Crystal on Silicon)-based SLM, as shown in Figure 1b; the MDP unit is composed of two LCoS-SLMs for the left and right eyes. Both LCoS panels can handle the resolution of full high definition (FHD, 1920 × 1080 pixels) with an effective diagonal size of 0.55 inches and pixel pitch of 6.3 $\mathsf{\mu }\mathrm{m}$ × 6.3 $\mathsf{\mu }\mathrm{m}$. In the holographic 3D display system, the image data of CGHs prepared by a master PC are delivered into the display module via a board of display control (BDC) as a control board, which has an interface of HDMI and USB ports that are directly connected to the PC. The MDP unit is connected to the BDC via 32-pin-FFC cables for data transfer. The BDC that we used in the study is a specifically designed control board to split an input image of resolution 2 × FHD into two independent images, each of which has the FHD resolution, and then transfer these two split images into independent LCoS-SLM panels, as depicted in Figure 1a. Figure 1b shows a photograph of the NEHD module for dual-view holographic content, in which a pair of amplitude-modulating hologram data are split to be uploaded on each LCoS-SLM for the left and right eyes.

2.2. Optical Components for the NHD System

We discuss optical components for the near-eye holographic display system, for which a schematic architecture is illustrated in Figure 2a. Collimated light from the green laser with 532 nm wavelength (CNI optoelectronics, MGL-S-532) is generated by using a first positive lens located just after the fiber-coupled laser source as a coherent illumination. To upload holographic 3D content, a flat LCoS-SLM panel that modulates an 8-bit grey-scaled amplitude is modified as a direct-viewing SLM type. First, the polarization direction of the illuminating light is parallel to that of a transparent, film-typed PBS (polarizing beam splitter), which is positioned at the input surface of the LCoS. Second, near the surface of the SLM, a second convex lens called a field lens is placed against the SLM to converge the output light toward the eye box of the observer. The active area of the LCoS-SLM is 12.1 mm (V) × 6.80 mm (H). It supports a pixel arrangement of a nearly rectangular-shaped matrix geometry. This pixel geometry generates an extension of the zeroth diffraction order in the Fourier plane, i.e., a virtual observation window (VW), which corresponds to the area of the eye box of the observer [10,16,17]. The size of VW along the i-axis (i = x, y) is approximately given by $W_i=\lambda \left(f/p_i\right)$, where $p_i$ is the pixel pitch of the SLM along the i-axis, f the focal length of the field lens, which is also related to the viewing distance of the observer from the SLM, and $\lambda$ the wavelength of illumination light. The size of the wavelength-dependent VW is designed to become larger than the pupil size of the human eye [18,19,20]. Thus, f = 500 mm is chosen for $\lambda$ = 532 nm to obtain $W_x$ = 42.2 mm (along the horizontal direction) and $W_y$ = 42.2 mm (along the vertical direction). The optical wave field emerging from a 3D object or scene can be directly observed within the region of VW with the help of both the transparent PBS film and the field lens, thus leading to holographic augmented reality (AR), enabling the viewer to experience genuine depth cues and an accommodation effect. Further, optical observation and an experimental evaluation of the optically reconstructed holographic 3D (H3D) images are performed near the area of VW at the eye position (see Figure 2a) and demonstrated by a camera-captured, holographic 3D (H3D) scene reconstructed from CGHs. As shown in Figure 2, the proposed near-eye holographic 3D display (NEHD) system consists of two split main components so that binocular observation can be achieved; after a collimated beam is split into two parts, its left-side part is designed to focus an incident beam towards the left-eye and the other part towards the right-eye.

Passing through each SLM, the light wave fields then are guided into two independent focal points corresponding to the two eyes’ observational positions. The separation between these two positions is equal to 65 mm in the eye plane of the observer, where the distance corresponds to the average binocular distance of a human [21,22] and is indicated as a binocular distance (BD) in Figure 2a.

2.3. Acquisition of 360° Multi-Viewed Holographic Content from a 3D Model

A depth map image, representing information on the distance between the surface of an object and the observation point in the unit of the greyscale, is useful to make 3D content modeling for hologram synthesis. It is necessary that a set of RGB color and depth map images per view be extracted from a 3D object, so that we may calculate CGH by using the algorithm of FFT-based cascaded Fresnel transform (see Equation (1)). In this study, instead of choosing specific camera sensors to capture depth information, such as Kinect or Real sensors, we use MAYA 2018^TM software [23] to design a 3D scene to give depth cues for holographic content. The software has the added advantage of rendering to capture the 3D object with a full range of 360 degrees. To experimentally observe the accommodation effect from holographic content, we design a 3D scene that consists of two Rubik’s cubes in green, each positioned at sufficiently. different distance to generate dynamic occlusion and to differentiate between focal depths.

Figure 3 shows the layout where two objects and a virtual camera are placed in the same plane, representing (a) a perspective view and (b) an in-plane view observed near the position of the camera eye. The camera can be programmed to orbit only along a virtual circular tract with the desired speed. While taking a turn on the circular orbit, the camera keeps shooting the 3D scene to obtain a total of 1024 frames, each shot with a constant increment of 0.352°, capturing a set of RGB images and depth maps simultaneously. Then both the RGB color and depth map images are stored in an 8-bit greyscale with BMP format. The distance from the center-of-mass of two objects to the camera’s orbit is measured and varied through a menu bar called a Distance Tool in MAYA^TM, and the depth value from the camera eye can then be manually adjusted by the user to determine the optimal depth range.

After the process of rendering in MAYA^TM is completed, a set of RGB and depth map images per frame, each holding FHD (1920 × 1080) resolution, is extracted. Thus, after a circular loop of the camera, the database where images are saved is composed of 1024 views. Through this process, we obtain RGB images and depth images of the target scene for generating 360° moving, holographic content.

2.4. Synthesis of CGH

The CGH synthesis for near-eye holographic content is explained on the basis of the wave optics and Fresnel’s holographic model in the system schematically illustrated in Figure 4 [10,11,12]. We assume that the distance ($D_1$) between the holographic display and the eye lens is equal to the focal length ($f_1$) of the field lens. When the size of eyeball and the focal length of eye lens are given as $D_2$ and $f_2$, we also assume that the hologram plane ($x_1$,$y_1$) near the field lens and the retinal plane ($x_2$, $y_2$) of the user are located in parallel with the eye lens plane (u,$\upsilon$). The optical field at the display panel with size of $X\times Y$ and that of retinal domain of the observer are denoted as $F\left(x_1, y_1\right)$ and $G\left(x_2, y_2\right)$, respectively. These fields are related to the following equation, called a cascaded Fresnel transform [24,25]:

$$G\left(x_2,y_2\right)=a\left(x_2, y_2\right)\iint \left[e^{\frac{i\pi }{\lambda }\left(\frac{1}{D_1}+\frac{1}{D_2}-\frac{1}{f_2}\right)\left(u^2+v^2\right)}\iint h\left(x_1,y_1,u,v\right)F\left(x_1,y_1\right)d{x}_{1}d{y}_1\right]\times e^{-\frac{2i\pi }{\lambda D_2}\left(x_{2}u+y_{2}v\right)}dudv,$$

(1)

where $a\left(x_2, y_2\right)=\frac{-e^{\frac{2i\pi \left(D_1+D_2\right)}{\lambda }}{e}^{\frac{i\pi \left(x_2^2+y_2^2\right)}{\lambda D_2}}}{D_1{D}_2{\lambda }^2}$, and $h\left(x_1,y_1,u,v\right)=rect\left(\frac{x_1}{X},\frac{y_1}{Y}\right)\times e^{\frac{i\pi }{\lambda D_1}\left(x_1^2+y_1^2\right)}{e}^{-\frac{2i\pi }{\lambda D_1}\left(x_{1}u+y_{1}v\right)}.$

While an image formed at the retinal plane is analyzed using the direct cascaded Fresnel transform, as given in Equation (1), making the desired image formation the optical field input at the display plane ($x_1$,$y_1$) is calculated using an inverse cascaded Fresnel transform. On the basis of Equation (1), the desired hologram field $H\left(x_1, y_1\right)$ in the display domain is synthesized after a process of superposition of a reference wave field with the initially calculated optical field $F\left(x_1, y_1\right)$. It is assumed that the reference wave field is the complex amplitude of the illumination wave from the backlight unit that is collimated, normally incident with a constant amplitude. In addition to the reference wave, we superimpose an off-axis carrier wave $C\left(x_1, y_1\right)$ on the calculated optical field $F\left(x_1, y_1\right)$ so that the desired signal function may shift from the noise terms, such as the DC term, and the twin-conjugate image. Through direct observation tests of each eye from H3D reconstructions, we experimentally derived the optimized conditions for the carrier wave format of $C\left(x_1, y_1\right)=e^{\left[\pi i\left(a{x}_1+b{y}_1\right)\right]}$, with a = 0.048 and b = 0.128, in applying the current-considered optical geometry (see Figure 4). Therefore, we could finally synthesize the signal function for the digital hologram $H\left(x, y\right)$; the signal function contains complex numbers in general, and can be expressed by $H\left(x, y\right)=\left|H\left(x, y\right)\right|e^{i\mathsf{\Phi }\left(x, y\right)}$, where $\left|H\left(x, y\right)\right|$ is the amplitude of $H\left(x,y\right)$ and $\mathsf{\Phi }\left(x,y\right)$ is its phase. Here, the CGH’s original function must be converted into an appropriate representation, that is, amplitude-modulating hologram data suitable for the study, because we use the amplitude-modulating SLM in our system.

2.5. Encoding Methods for the Amplitude-Modulating Device

In this section, we first explore the established approaches to convert the complex-valued hologram to express the amplitude-modulating SLM (AM-SLM). We then propose our novel encoding scheme, i.e., the modified amplitude-only encoding (MAO) approach.

Among methods to circumvent the non-availability of commercial complex-modulating SLMs, a conventional amplitude-only encoding (CAO) [16] has been used. The CAO approach is to extract only the real part of a complex-valued optical field at the pixel coordinates of the hologram plane, and subsequently to bias the real-valued space into nonnegative, real-valued space by level shifting, which can be written as

$$H_{CAO}\left(x, y\right)=H_R\left(x, y\right)+H_{off},$$

(2)

where $H_R\left(x, y\right)=\left|H\left(x, y\right)\right|\cos \left[\mathsf{\Phi }\left(x, y\right)\right]$ is the real part of the original complex field, and $H_{off}$ is an offset to make $H_{CAO}\left(x, y\right)$ non-negative and is equal to the absolute value of a minimum of $H_R\left(x, y\right)$ for the CAO, i.e., $H_R=\left|min\left\{H_R\left(x, y\right)\right\}\right|$.

On the other hand, as an alternative representation format of computer-generated holograms (CGHs) available for the AM-SLM, there are two kinds of effective solutions from an analytic perspective: The first analytic representation scheme is Lee’s method, which decomposes a complex-valued field into four real and non-negative components [13]. The decomposition by Lee’s method is expressed as

$$H_L\left(x, y\right)=L_1\left(x, y\right)e^{i0}+L_2\left(x, y\right)e^{i\pi /2}+L_3\left(x, y\right)e^{i\pi }+L_4\left(x, y\right)e^{i3\pi /2},$$

(3)

where at least two among four coefficients $L_m\left(x, y\right) \left(m=1~4\right)$ are equal to zero. The relative phase values for Lee’s method are physically implemented by lateral displacement within a single macro-pixel that consists of four sub-pixels. Alternatively, Burckhardt’s method, which is a simplified version of Lee’s method, uses the feature that any complex-valued function can be analytically decomposed into three real and nonnegative components. The decomposition by Burckhardt’s method can be expressed by

$$H_B\left(x, y\right)=B_1\left(x, y\right)e^{i0}+B_2\left(x, y\right)e^{i2\pi /3}+B_3\left(x, y\right)e^{i4\pi /3},$$

(4)

where at least one among the three coefficients $B_n\left(x, y\right) \left(n=1~3\right)$ is equal to zero [14,15]. The relative phase values for Burckhardt’s method are realized by lateral displacement within one macro-pixel that comprises three sub-pixels.

Finally, the equational representation of the modified amplitude-only encoding (MAO) method we propose is described in the following formula, provided that the minimum of $H_R\left(x, y\right)$ is negative:

$$H_{MAO}\left(x, y\right)=H_{int}\left(x, y\right)-min\left\{H_{int}\left(x, y\right)\right\}$$

(5)

where $H_{int}\left(x, y\right)$ is an intermediate function and is defined by

$$H_{int}\left(x, y\right)=\left\{\begin{array}{c}{H}_R\left(x, y\right) for H_R\ge 0,\\ \frac{\epsilon H_R\left(x, y\right)}{min\left\{H_R\left(x, y\right)\right\}} for H_R<0.\end{array}\right.$$

(6)

This formula of the MAO scheme has two key features: (i) For the pixel of $H_R\left(x, y\right)\ge 0$, the intermediate function $H_{int}\left(x, y\right)$ is not changed from $H_R\left(x, y\right)$ and retains the original form of $H_R\left(x, y\right)$. The result of MAO encoding is thus equivalent to that of conventional amplitude-only encoding, that is, $H_{MAO}\left(x, y\right)=H_{CAO}\left(x, y\right)$. (ii) For the pixel of $H_R\left(x, y\right)<0$, a scale-down transform process is first performed before $H_R\left(x, y\right)$ is divided by the minimal value of $H_R\left(x, y\right)$. Its effect reduced with a fixed rate is determined by a parametric factor $\mathsf{\epsilon }$, where $0<\mathsf{\epsilon }\ll 1$ is assumed. When we prepare the intermediate function $H_{int}\left(x, y\right)$, we finally obtain $H_{MAO}\left(x, y\right)$ just after subtracting $H_{int}\left(x, y\right)$ by its minimum [see Equation (5)]. Using the above-explained approaches, we establish the framework that consist of four different kinds of amplitude holograms, each of which is converted from the original, complex-valued hologram calculated by the FFT algorithm and is prepared for experimental comparison. Figure 5 shows the results of each encoding using the same original CGH function generated from a set of RGB and depth map images as shown in Figure 3c,d, with the MAO encoding processed under the condition of $\mathsf{\epsilon }=1/50$.

For the observer to experience an enhanced three-dimensional effect, two different viewpoints about a given 3D scene are required. Hence, for the dual-view holographic display, we made a set of CGH frames by selectively sorting two images per CGH frame among 1024 prepared images to provide 360° full views of the scene. We now explain that the viewing-point difference per CGH frame has a gap of 8 numbers from a series of 1024 views with a time-sequential difference. For example, the first view image for the left eye and the ninth view image for the right eye can be chosen as the first CGH content for a single frame. The reason is as follows: when the radius of the circular sector, i.e., the distance between the eyes’ plane and the object’s plane, is 1.4 m, the arc, i.e., the average distance (BD) between the eyes, is 65 mm [21,22], and thus their central angle should be 2.66°. Because the central angle increment per image is 0.352°, the view-number difference between the view of left eye and the view of right eye is equal to 8. A brief recapitulation of preparing CGH content fit with the NEHD system is as follows: (i) We obtained two complex hologram matrices through the FFT algorithm from the first set that consists of one RGB-depth image for the left eye and the other for the right eye (see Figure 6). (ii) Then we computed the four types of CGH content by using CAO, MAO, Lee’s, and Burckhardt’s encoding methods. For each encoding, a pair of calculated CGHs are combined in a side-by-side arrangement and stored as an image file with a resolution of 3840 × 1080. Figure 6 shows an exemplary process in the case of MAO encoding to prepare the first CGH frame for our dual-view holographic display. Here, we chose two sets of RGB-depth images with different viewpoints; one is for the first viewpoint and the other is for the ninth viewpoint. (iii) We repeated the above process about the 360° full range to create the final holographic content fit for the binocular geometry. (iv) Finally, we converted the time-sequential 360° views’ CGH set into an uncompressed video file, which can show a holographic 3D movie with a speed of 30 frames per second (fps) [see the Supplementary video (a)~(d)].

3. Experimental Results

3.1. Experimental Conditions

Since the simulation results can visualize a clear accommodation effect from the CGH image, we first tested observations derived from numerical computation by directly using the encoded CGH database prepared via the aforementioned process. Figure 7 shows the numerically reconstructed images with a focus on the fore-positioned object in the cases of CAO encoding (a), MAO encoding (b), Burckhardt’s encoding (c), and Lee’s encoding (d). Another example of numerical reconstruction under the same conditions, except focused on the back-positioned object, is shown in Figure 8. Through these simulations, we could expect that the four encoded kinds of CGH content will support both the depth cue and accommodation effect. The experimental observations of optical reconstruction from these four kinds of encoded holograms were taken in a darkroom by using a DSLR camera (Canon EOD 5D Mark III and Canon EF 70–200 mm F2.8 IS USM Lens). Photographs of holographic reconstructions were captured with the camera’s lens positioned near the focal position of the field lens, i.e., the eye position of the observer at the VW (see Figure 2a). The camera settings for this experiment are as follows: shutter speed is 1/30 s, aperture size f 2.8, ISO 8000, and the focal length of the optical lens (Canon EF 70–200 mm F2.8) attached to the camera to take the pictures is about 1.45 m. Figure 9 and Figure 10 show photographs of optically reconstructed scenes obtained by the NHD, which displays four kinds of encoded holograms and which manifest clear depth discrimination.

We observed that background light and speckle patterns prominently appear in the 3D scene reconstructed from the CAO-encoded hologram, as shown in Figure 9a. On the other hand, we found that non-diffracted background light is efficiently suppressed in the holograms of the other three encoding methods (see Figure 9b–d). We noticed that the diffraction efficiency of Burckhardt’s hologram and Lee’s hologram are much lower than that of the MAO hologram. Furthermore, Figure 9b indicates that both background noise and speckle noise are dominantly attenuated in the case of MAO encoding in comparison with the rest of the encodings. Figure 11 shows the comparison result with a surface intensity map that plots the surface of the grey-level, $\mathrm{g}\left(x_i, y_i\right)$, according to each encoding within the bright area to be scanned on a selected, focused object. We supposed that an object with an ideally speckle-free region has a surface intensity map with a regularly uniform feature. Hence, as an alternative approach for a speckle-noise analysis, the work to plot the surface intensity of the holographically reconstructed object indicates that better uniformity corresponds to less speckle noise. Through an examination of the surface-intensity indicator (see Figure 11), we found that the surface quality of the H3D scene regenerated by MAO encoding is the best among the four different encodings.

For a quantitative evaluation of the image quality of the holographic 3D reconstruction, we select a scanned measurement area with the same region on photographic images to capture the optically reconstructed H3D scenes with respect to four different encodings. The scanned test area that we chose for each encoding consists of 27 × 38 pixels, as shown in Figure 9. By a statistical analysis within the scanned areas, we calculate the corresponding holographic 3D image’s brightness (HIB) and holographic contrast ratio (HCR). Here, we define the HIB as the peak value of a grey-level curve along each selected line of the scanned areas. The HCR is defined as

$${\mathrm{HCR}}_{ij}=\left(G_{Wi}-G_{Oi}\right)/\left(G_{Bj}-G_{Oj}\right),$$

(7)

where $G_{Wi}$ and $G_{Oi}$ represent the grey level and the dark-noise level (compensation term of $G_{Wi}$), respectively, measured at a target pixel point $\left(x_i, y_i\right)$ in the bright state condition. $G_{Bj}$ and $G_{Oj}$ represent the grey level and the dark-noise level (compensation term of $G_{Bj}$), respectively, measured at a target pixel point $\left(x_j, y_j\right)$ in the black state condition. In general, the HCR value is equal to the result of dividing the representative grey-value of the bright object area by that of the dark background area on the given holographic 3D reconstruction. In order to calculate the HCR, each representative value of the bright area and dark area is required. In the study, we take a size of 27 × 38 pixels (row × column) as a rectangular sample area within the bright/dark region for the HCR.

The sampling areas in Figure 9 are marked with white lines, but all pixels’ values of the rectangular area are taken for the statistical analysis. We set the mean value of grey levels within the scanning area as a representative value because the measuring unit of the brightness (HIB) in the holographic 3D experiment is 8-bit greyscale. Figure 12a,b show the brightness distribution of the bright and dark regions by using the boxplot method. Here, the blue asterisk represents the mean (average) value of the samples and the red cross represents the median value of the samples.

3.2. Results and Analysis

We evaluated the performance of the hologram-encoding methods based on the aforementioned three image quality measurement metrics, average holographic 3D image’s brightness (HIB), holographic contrast ratio (HCR), and holographic uniformity of surface (HUS). For the performance evaluation, we set each rectangular area sample (unit area’s size: 27 × 38 pixels) of a bright state area and dark state area in the reconstructed H3D scene as the scanning region (see Figure 9). Figure 11 shows a typical surface intensity map in the bright state area of a reconstructed H3D scene according to each encoding method. Figure 12 shows the box-whisker plot that maps the surface intensity of bright/dark area samples to be scanned on each H3D scene optically reconstructed from four encoding types. From both the holographic surface uniformity (HUS) characteristic and the holographic intensity (HIB) measurement in the bright areas, we first found that the surface uniformity of H3D reconstruction is highest in the order of MAO, LEE/BUR, and CAO. Moreover, from the HIB test using samples of bright areas, we found that although the average brightness of MAO is about 10.4% lower than that of CAO, MAO’s average brightness is about 11.6% higher than that of BUR or LEE. Second, Figure 13 presents a HCR-HIB diagram of the optically reconstructed H3D scenes with regard to the four encoding methods. The average brightness of Burckhardt’s and Lee’s encoding methods is approximately 19.6% lower than that of CAO encoding. However, the HCR values of these two encodings are at least two times higher than that of the CAO encoding. Above all, the diagram shows that the HCR of the MAO method is three times greater than that of the CAO method at the cost of only a 10% decrease in brightness in comparison with the brightness of the CAO encoding. Furthermore, the HCR of the MAO method is still 30% higher than that of Burckhardt’s encoding or Lee’s encoding. Thus, the analysis through the HCR-HIB diagram and the brightness graph in Figure 12a indicates that the MAO approach proposed in this study not only provides the most effective eliminator of DC background noise but also achieves refined performance in holographic contrast ratio (HCR) in comparison to Burckhardt’s and Lee’s methods.

Third, we also assessed the surface uniformity of the reconstructed H3D object in brightness distribution by the method of a box-whisker plot, where a parameter of IQR (Inter Quartile Range) is defined as the range of 50% data centered on a median value, equal to each box’s length at the boxplot; the smaller the IQR value is, the more uniform the surface of a holographic reconstructed object is. Through observation of each sample in the local bright area, we found that CAO’s IQR is 20.5, MAO’s IQR is 14, Burckhardt’s IQR is 9, and Lee’s IQR is 11. The results are derived from consideration of only 50% of the surface intensity data, and thus Burckhardt’s encoding has the smallest IQR. In addition, we checked the characteristic behavior of outliers in the box-whisker plot; in fact, all encoding methods except for the MAO encoding case have outlier values, where data of brightness are positioned out of the boxplot’s whisker (see Figure 12a). CAO encoding has 20 outliers, Burckhardt’s encoding has 23 outliers, and Lee’s encoding has 14 outliers, whereas MAO encoding has no outlier. Here, it is necessary to note that these outliers correlate with the extent of intensity irregularity in the measured local surface. A wider fluctuation of the intensity distribution means that the chosen surface in the given holographic 3D scene has less uniformity. Thus, both the IQR and the outliers of the boxplot indicate that the intensity distribution of MAO encoding has the narrowest range of spectrum among the four kinds of encodings within the scanned bright area, which is a convenient guides by which to measure the quality of surface homogeneity on the given holographic 3D reconstruction.

4. Conclusions

In conclusion, a novel near-eye holographic 3D (H3D) display prototype, applicable to holographic AR/MR devices, is proposed and experimentally demonstrated with 360° full-viewed holographic content to be optically reconstructed as an H3D movie at 30 fps. To achieve a high quality of optical H3D reconstruction, we also proposed an advanced amplitude-modulating encoding scheme (MAO encoding) that is well suited for AM-SLMs, and experimentally verified, for the first time, that the proposed scheme can clearly improve the image quality of monochromatic H3D reconstruction, through quantitative statistical analyses, in comparison with other competitive encodings such as Burckhardt’s encoding and Lee’s encoding. The presented techniques for NEHD we proposed enable an observer to experience the depth cue, a realistic accommodation effect, and high-quality holographic 3D movies at each eye of the user.

A restriction on the NEHD prototype demonstrated in this study is to cover the monochromatic H3D reconstruction. For the full-color implementation of the NEHD, it is necessary to develop RGB color-matching technologies in real space, including the optical component modification of RGB waveguiding and the RGB image processing of CGHs. Additionally, a field-sequential full color implementation within the presented operation schemes will be accomplished as the next step in the near future. The scopes that we demonstrated in this study did not include searching for the optimum of the parametric factor ($\mathsf{\epsilon }$) on the H3D image reconstructed from the proposed novel MAO encoding. For the parametric scaling factor (ε), we chose between 1/50 and 1/100, leading to the same behavior results under the sensitivity limit of the DSLR camera we used for H3D image monitoring. To find out the optimal condition of the (ε) factor remains another research work, which could be performed by using highly precise tool-based apparatus and measuring techniques. Holographic content treated in this research consists only of 3D objects from a pair of cube shapes without a background image. Thus, to support a hyper-realistic impact on XR environments such as the holographic AR/MR and metaverse applications, researchers are trying to combine holographic 3D object content with a variety of background images.

We expect to improve the proposed NEHD system towards commercial XR realization through a series of further studies, including the optimization of the parametric factor ($\mathsf{\epsilon }$) on the real-like H3D scenes reconstructed from the MAO encoding approach and the holographic measure metrics which we define in the study.