Fast 3D Analytical Affine Transformation for Polygon-Based Computer-Generated Holograms

Abstract

We present a fast 3D analytical affine transformation (F3DAAT) method to obtain polygon-based computer-generated holograms (CGHs). CGHs consisting of tens of thousands of triangles from 3D objects are obtained by this method. We have attempted a revised method based on previous 3D affine transformation methods. In order to improve computational efficiency, we have derived and analyzed our proposed affine transformation matrix. We show that we have further increased the computational efficiency compared with previous affine methods. We also have added flat shading to improve the reconstructed image quality. A 3D object from a 3D camera is reconstructed holographically by numerical and optical experiments.

1. Introduction

With the rapid development of display and computer technology, 3D display technology has made great progress [1,2]. The current 3D display technologies generally include binocular parallax display [3], volume 3D display [4], light field display [5] and holographic 3D display. Holographic 3D display is a technology to reconstruct 3D wave-front by using light wave diffraction information based on the principle of wave optics. Because the wave-front information of the 3D scene is reconstructed, it can provide all the depth cues required by the human eye [6,7,8]. The wave-front recording process of holography can be completed and simulated by a computer to generate the so-called computer-generated holograms (CGHs). The use of CGHs avoids the setup of complicated optical paths in optical holography [9,10], as long as the mathematical description of the 3D scene is obtained and transformed into the wave-front distribution in the hologram plane by algorithms. Therefore, the algorithm of calculating CGHs is key, because it directly determines the computational efficiency of the hologram and the quality of the reconstructed image [11,12].

Indeed, the mathematical description of a 3D scene can also be expressed in many forms. According to the geometric information of its surface, it can be discretely expressed as a collection of point sources, giving the so-called point-based method [13]. The wavefront information of the whole 3D object is obtained by calculating and then adding the light field distribution of the spherical wave emitted by each point light source in the hologram plane. However, the required number of point sources is usually as high as millions, leading to computational bottlenecks. By using a look-up tables (LUTs) [14,15,16] and graphic processing units (GPUs) [17,18], we can accelerate the calculations greatly. Another popular CGH calculation method is the use of polygons (usually represented by triangle meshes) to approximate the surface of a 3D scene, giving to the polygon-based method [19,20,21,22,23,24,25,26,27,28,29,30,31,32]. Compared with the point-based method, the number of polygons can be greatly reduced, and the mature theory of computer graphics can be used to make the reconstructed scene more realistic. There are other methods to decompose 3D objects, such as the layer-based [33,34] or line-segment based method [35].

In polygon-based methods, the propagation of light field can be calculated for each polygon by using the angular spectrum theory if the polygon is parallel to the hologram plane [6]. The essence of polygon-based methods is the diffraction calculation between non-parallel planes. Polygon-based methods can be divided into two categories; one is the traditional method based on sampling [8]. This method needs to sample each tilted polygon that is not parallel to the hologram in the spatial and frequency domains. Due to the polygon rotation, sampling in the spatial and frequency domains creates uneven sampling intervals between the two domains. Time-consuming interpolation is needed to alleviate sampling distortion. The other method is the analytical method [8]. Unlike the traditional method, sampling in the spatial domain to obtain the spectrum of a tilted polygon is not needed. Instead, the spectrum of a tilted polygon can be expressed in terms of the spectrum of a unit triangle (also called primitive triangle), which is known analytically, through the 2D affine transform [23]. Therefore, only sampling is needed in the frequency domain for each polygon, bypassing the need for interpolation in the traditional method. Compared with the traditional method, the analytical method effectively reduces the amount of calculations. However, shaping and texture mapping are not easily included in the analytical methods as compared with the traditional methods [8].

Pan et al. [25,26] have developed an analytical method utilizing 3D affine transformation. They have defined a pseudo inverse matrix to map the spatial triangle with a three-dimensional right primitive triangle (which has an analytical spectrum expression). The major issue of the technique is the inaccuracy caused by the inversion of the pseudo matrix. Zhang et al. [28,29] have derived a correct analytical expression in the context of 2D affine transformation [23] and proposed a method to achieve 3D transformation from an arbitrary triangle to a primitive triangle through a 3D rotation of the arbitrary triangle and the use of a 2D affine matrix [29]. Although this method avoids the time-consuming calculation of the pseudo inverse matrix, the process is rather complex. Zhang et al. [8] also have proposed a method called the fast 3D affine transformation (F3DAT) method by translating the primitive triangle to avoid the use of the pseudo inverse matrix to improve the computational efficiency.

In the wave-optics based approach, it can provide accurate depth cues. However, view-dependent properties’ rendering requires additional calculations. Rendering technology of computer graphics makes the reconstructed 3D scene more realistic [36,37,38]. Matsushima et al., have discussed the methods of shading and texturing [20,21,22] and created a large-scale full-color CGH successfully [39]. Subsequently, many methods have been proposed to improve the quality of reconstructed images through texturing [36,37], shading [22,38], and resolution improvement [40]. Additionally, the use of the silhouette method for hidden surface removal has been described [41].

In this study, based on the three-dimensional affine theory [25,29], we present a fast 3D analytical affine transformation (F3DAAT) method to obtain a full-analytical spectrum of a spatial triangle. We obtain the analytical expression of a 3D affine matrix algebraically, and the spectrum of tilted triangles can be obtained directly. Compared with previous methods, we show improved computational efficiency. In addition, in order to improve the image quality, we add flat shading to make the reconstructed image more realistic. We also demonstrate reconstructed 3D objects composed of tens of thousands of polygons numerically as well as the use of a spatial light modulator (SLM) for optical reconstruction.

In Section 2, we briefly introduce the basic principle of the polygon-based method. In Section 3, we present the theory of F3DAAT. In Section 4, we demonstrate the reconstruction of 3D objects numerically and optically. The computational efficiency is also compared with previous methods. The results of adding flat shading are also illustrated.

2. Conventional Polygon-Based Method

Figure 1 shows the basic principle of the polygon-based method. The surface of a 3D object is discretized into many polygons (usually triangles). The hologram is assumed to be on the plane $z=0$. The total complex field distribution on the hologram plane $f_{HOLO}\left(x,y,z=0\right)$ can be expressed as the superposition of the polygon fields from each polygon, $f_{HOLO,i}\left(x,y\right)$:

$$f_{HOLO}\left(x,y\right)=\sum _{i=1}^N{f}_{HOLO,i}\left(x,y\right),$$

(1)

where $N$ is the number of polygons and $f_{HOLO,i}\left(x,y\right)$ is the complex field on the hologram plane from the $i\mathrm{th}$ polygon. Since the complex field distribution of each triangle on the hologram can be expressed by the inverse Fourier transform of its spectrum:

$$f_{HOLO,i}\left(x,y\right)={ℱ}^{-1}\left\{F_{HOLO,i}\left(u,v\right)\right\},$$

(2)

where ${ℱ}^{-1}\{·\}$ represent the inverse Fourier transform and $F_{HOLO,i}\left(u,v\right)$ represents the spectrum of the $i\mathrm{th}$ polygon on the hologram. Therefore, the key to the polygon-based method is how to obtain the spectrum of each triangle on the hologram plane.

Figure 1. Principle of the polygon-based method.

As shown in Figure 2, a polygon on the source coordinate system $\left(x_s,y_s,z_s\right)$ is not necessarily parallel to the hologram plane $\left(x,y\right)$; one needs to rotate the polygon to parallel local coordinate system $\left(x_p,y_p,z_p\right)$ to be parallel to the hologram plane in order to calculate the diffracted field toward the hologram through standard diffraction theory [6]. The spatial frequencies $\left(u_s,v_s\right)$ can be expressed as:

$$u_s=a_1{u}_p+a_2{v}_p+a_3{w}_p,$$

(3a)

$$v_s=a_4{u}_p+a_5{v}_p+a_6{w}_p,$$

(3b)

where $a_i$ are the elements of the rotation matrix. $\left(u_s,v_s\right)$ (corresponding to $\left(k_{sx},k_{sy}\right)$ in Equation (10a,b) in Ref. [8]) are the spatial frequencies corresponding to source coordinates $\left(x_s,y_s,z_s\right)$. $\left(u_p,v_p,w_p\right)$ (corresponding to $\left(k_{px},k_{py},k_{pz}\right)$ in Equation (10a,b) in Ref. [8]) are spatial frequencies corresponding to parallel local coordinates $\left(x_p,y_p,z_p\right)$. Upon a differential operation in Equation (3a), we have:

$$\mathsf{\Delta }{u}_s=a_1\mathsf{\Delta }{u}_p+a_2\mathsf{\Delta }{v}_p+a_3\mathsf{\Delta }{w}_p=a_1\mathsf{\Delta }{u}_p+a_2\mathsf{\Delta }{v}_p-a_3·\frac{\left(u_p·\Delta u_p+v·\Delta v_p\right)}{\sqrt{\frac{1}{{\lambda }^2}-{u_p}^2-{v_p}^2}},$$

(4)

where $w_p=\sqrt{\frac{1}{{\lambda }^2}-{u_p}^2-{v_p}^2}$ with $\lambda$ being the wavelength of the light source. Since the spatial frequencies $\left(u_p,v_p\right)$ are uniformly distributed, we have $\Delta u_p=constant, \Delta v_p=constant.$ For simplicity, let us assume $\mathsf{\Delta }{u}_p=1,\mathsf{\Delta }{v}_p=1,$ and Equation (4) then can be rewritten as follows:

$$\mathsf{\Delta }{u}_s=a_1+a_2-a_3·\frac{\left(u_p+v_p\right)}{\sqrt{\frac{1}{{\lambda }^2}-{u_p}^2-{v_p}^2}}\ne constant.$$

(5)

Figure 2. Coordinate systems of the traditional polygon-based method: source coordinate system $\left(x_s,y_s,z_s\right)$, parallel local coordinate system $\left(x_p,y_p,z_p\right)$ and hologram plane $\left(x,y\right).$ Adapted from Zhang et al. [8].

Since $\mathsf{\Delta }{u}_s$(the derivative of $u_s$) is not constant, Equation (5) indicates that for an arbitrary set of spatial frequencies $\left(u_p,v_p\right)$ corresponding to the parallel local coordinates, spatial frequency $\left(u_s,v_s\right)$ after rotation has highly nonlinear properties. Because uniform sampling is necessary for the FFT to work correctly in the traditional method, interpolation process is required for spatial frequencies $\left(u_s,v_s\right)$ after rotation. This procedure would add substantially to the computational time [30]. Comparing with the traditional method, the analytical method avoids the use of the FFT in obtaining the spectrum of the polygon in the source coordinates and the interpolation process is therefore not required. The analytical method can obtain the analytical expression of the tilted triangle spectrum directly by using affine transformation and a given spectrum expression of a primitive triangle. In this paper, we analytically solve the affine matrix, which avoids the most time-consuming steps in previous methods [8,25,29] and further improves the computational efficiency over the previous methods.

3. Theory

The aim of the polygon-based method is the calculation of the polygon field in the plane of the hologram. However, we cannot obtain the polygon field or its spectrum directly by using standard diffraction theory, which is valid only between parallel planes. One of the conventional approaches to solve this problem is to map the desired light field or its spectrum using affine relations. The traditional affine transformation method is based on procedures such as rotation and translation to establish the relationship between the input and output coordinates, with the output represented by a set of known inputs and affine relations. Hence, the essence of affine transformation method is a mapping method, and the core problem of affine transformation is to find the affine matrix. In our proposed theory, we will find the affine matrix algebraically in a universal way.

In the traditional 3D affine transformation algorithm, there is a global coordinate system $\left(x,y,z\right)$ as the output coordinate system, as shown in Figure 3. The hologram plane $\left(x,y\right)$ is located in $z=0$, and the tilted triangle $\mathsf{\Pi }$ with vertexes $\left(x_1,y_1,z_1\right),\left(x_2,y_2,z_2\right)$ and $\left(x_3,y_3,z_3\right)$ is located in the global coordinate system.

Figure 3. Tilted triangle $\mathsf{\Pi }$ in global coordinate system $\left(x,y,z\right)$ and hologram plane $\left(z=0\right).$

We now define the affine coordinates $(s,t,p$) as our input coordinates, as shown in Figure 4. A primitive triangle $\mathsf{\Delta }$ with vertexes $\left(s_1,t_1,p_1\right),\left(s_2,t_2,p_2\right)$ and $\left(s_3,t_3,p_3\right)$ is in the affine coordinate system, and the analytical expression of the primitive triangle spectrum is obtained by using two-dimensional Fourier transform for $p=0$. The spectrum of the tilted triangle is finally obtained from the affine relationship between the primitive triangle $\mathsf{\Delta }$ and the tilted triangle $\mathsf{\Pi }$ with the analytical expression of the primitive triangle spectrum derived from the Fourier transform.

Figure 4. Primitive triangle $\mathsf{\Delta }$ in affine coordinate system $\left(s,t,p\right)$.

Through affine transformation $\overrightarrow{{r_g}^t}=R\overrightarrow{{r_a}^t}+\overrightarrow{C },$ we can map affine coordinates $\left(s,t,p\right)$ into global (output) coordinates $\left(x,y,z\right)$, where $\overrightarrow{{r_g}^t}={\left(x,y,z\right)}^t$ is the coordinate vector of tilted triangle $\mathsf{\Pi }$ in the global coordinate system, $\overrightarrow{{r_a}^t}={\left(s,t,p\right)}^t$ is the coordinate vector of primitive triangle $\mathsf{\Delta }$ in the affine coordinates and the superscript $t$ denotes the transpose operation. $R$ is a 3 × 3 matrix and $\overrightarrow{C}$ is a 3 × 1 vector. We let $\left(s_1,t_1,p_1\right)$ =$\left(0,0,0\right)$,$\left(s_2,t_2,p_2\right)$ =$\left(1,0,0\right) \mathrm{and} \left(s_3,t_3,p_3\right)$ =$\left(0,1,0\right)$. Therefore, we can write in terms of matrix multiplication as shown in Equation (6), where $T$ is the affine transformation matrix and $G$ and $A$ represent matrices consisting of vertexes of tilted triangle $\mathsf{\Pi }$ and primitive triangle $\mathsf{\Delta }$, respectively. By calculating the twelve parameters of affine matrix $T$, we can uniquely determine the 3D affine transformation:

$$G=\left[\begin{array}{ccc}{x}_1& x_2& x_3\\ y_1& y_2& y_3\\ z_1& z_2& z_3\\ 1& 1& 1\end{array}\right]=TA=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& c_1\\ r_{21}& r_{22}& r_{23}& c_2\\ r_{31}& r_{22}& r_{33}& c_3\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}{s}_1& s_2& s_3\\ t_1& t_2& t_3\\ p_1& p_2& p_3\\ 1& 1& 1\end{array}\right].$$

(6)

The key to the affine transformation algorithm is to find the elements of affine matrix $T$. There is no inverse for 4 × 3 matrix $A$, because inverse only exists for square matrices. The affine matrix $T$ has been found by using the pseudo inverse matrix of $A$ [25]. The accurate method is to avoid the use of pseudo matrices and to find the affine transformation matrix $T$ through direct calculation of $T=G{A}^{-1}$. There are twelve unknown elements in affine matrix $T$, and so we have to solve twelve equations to get these twelve elements. However, each matrix $G$ and $A$ only contain three vertexes in Equation (6). From this, we can only get nine equations to solve for the nine elements; the remaining elements in the affine matrix $T$ cannot be determined. In light of this, we introduce a new set of vertexes ${\left(s_4,t_4,p_4\right)}^t$ in matrix $A$ and a 4 × 1 vector ${\left[d_1,d_2,d_3,d_4\right]}^t$ in matrix $G$ to determine all the unknown twelve elements of the affine matrix $T$. Therefore, we extend $G$ and $A$ to a $4\times 4$ matrix $G_1$ and $A_1$, respectively. In light of this, Equation (6) becomes:

$$G_1=\left[\begin{array}{cccc}{x}_1& x_2& x_3& d_1\\ y_1& y_2& y_3& d_2\\ z_1& z_2& z_3& d_3\\ 1& 1& 1& d_4\end{array}\right]=T{A}_1=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& c_1\\ r_{21}& r_{22}& r_{23}& c_2\\ r_{31}& r_{22}& r_{33}& c_3\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{s}_1& s_2& s_3& s_4\\ t_1& t_2& t_3& t_4\\ p_1& p_2& p_3& p_4\\ 1& 1& 1& d_4\end{array}\right].$$

(7)

In order to simplify calculations and improve the calculation efficiency, the best choice is let $d_1=d_2=d_3=d_4=0$ and $\left(s_4,t_4,p_4\right)$ =$\left(0,0,1\right)$. Then, $A_1$ becomes

$$A_1=\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\\ 1& 1& 1& 0\end{array}\right].$$

(8)

Through simple calculations, we can obtain $\det \left(A_1\right)=-1$, indicating the existence of the inverse matrix of $A_1.$ Then, we can find the affine transformation matrix $T$ by $T={G_1{A}_1}^{-1}$. Note that the vertex ${\left(s_4,t_4,p_4\right)}^t$ only represents a point located in the primitive coordinate system independent of the primitive triangle, and the value of ${\left(s_4,t_4,p_4\right)}^t$ and $d_4$ must be chosen such that $\det \left(A_1\right)\ne 0$ to make sure the inverse of $A_1$ exists. $d_{1~3}$ can take any value; different values of $d_{1~3}$ make a different affine transformation matrix $T$ and the Jacobian determinant also changes at the same time. Again, we have chosen Equation (8) for ease of calculations and there is no effect on the final results that we end up solving. According to the above discussion, the affine transformation matrix $T$ can be obtained by Equation (7):

$$T={G_1{A}_1}^{-1}=\left[\begin{array}{cccc}-x_1+x_2& -x_1+x_3& 0& x_1\\ -y_1+y_2& -y_1+y_3& 0& y_1\\ -z_1+z_2& -z_1+z_3& 0& z_1\\ 0& 0& 0& 1\end{array}\right].$$

(9)

For primitive triangle $\mathsf{\Delta }$, assuming that its surface function has strength of one within the triangle and zero outside, its spectrum analytical expression $F_{\Delta }\left(u_s,v_t\right)$ can be obtained (see Appendix A):

$$\begin{gathered} F_{\Delta }\left(u_s,v_t\right)& ={\iint }_{\mathsf{\Delta }}1·e^{-j2\pi \left(u_{s}s+v_{t}t\right)}dsdt \\ & =\frac{1}{2{\pi }^2}\left[\frac{{\mathrm{e}}^{-j2\pi v_t}}{\left(u_s-v_t\right)\cdot v_t}-\frac{{\mathrm{e}}^{-j2\pi u_s}}{\left(u_s-v_t\right)\cdot u_s}-\frac{1}{u_s\cdot v_t}\right], \end{gathered}$$

(10)

where $u_s,v_t$ are the spatial frequencies corresponding to affine coordinates $s$ and $t$. The singular points $u_s=0$, $v_t=0$ or $u_s-v_t=0$ have been discussed in Appendix A. Through affine transformation, the spectrum distribution of tilted triangle $\mathsf{\Pi }$ on the hologram plane is (see the derivation of this equation in Appendix B):

$$\begin{gathered} F_{HOLO}\left(u,v\right)={\iint }_{\mathsf{\Pi }}f\left(\overrightarrow{r_g}\right)·{\mathrm{e}}^{-j2\pi \left(ux+vy+wz\right)}dxdy \\ ={\iint }_{\mathsf{\Delta }}f(R\overrightarrow{r_a}+\overrightarrow{C})·{\mathrm{e}}^{-j2\pi \left(ux+vy+wz\right)}·J·dsdt, \end{gathered}$$

(11)

where $u,v,w$ again are spatial frequencies of the global coordinate system, $f\left(\overrightarrow{r_g}\right)=f\left(x,y,z\right)$ is the surface function of the tilted triangle $\overrightarrow{r_g}$=$\left(x,y,z\right)$, $\overrightarrow{r_a}$=$\left(s,t,p\right)$ and the Jacobian determinant $J=r_{11}{r}_{22}-r_{12}{r}_{21}$. Then, the spectrum distribution on the hologram $F_{HOLO}$ is:

$$F_{HOLO}\left(u,v\right)=J·{\mathrm{e}}^{-j2\pi \left(u{x}_1+v{y}_1+w{z}_1\right)}·F_{\Delta }\left(u_s,v_t\right),$$

(12)

which has been derived in Appendix C. Note that for an arbitrary tilted polygon, we can set an arbitrary point instead of point $\left(x_1,y_1,z_1\right)$ of the polygon. For a different choice, it will change the affine transform matrix, but the Jacobian determinant in Equation (12) also changes at the same time, giving the same result in Equation (12).

As shown in Figure 5a, the zero frequency in the global coordinate system is ${\left(u_0, v_0, w_0\right)}^t={\left(0, 0, \frac{1}{\lambda }\right)}^t$. According to the spatial frequency relationship given by Equation (28) in Appendix C, as shown in Figure 5b, the zero frequency in the affine coordinate system ${\left(u_{s0},v_{t0},w_{p0}\right)}^t$ is ${\left(u_{s0},v_{t0},w_{p0}\right)}^t=R^{-1}{\left(u_0,v_0,w_0\right)}^t$:

$$\left(\begin{array}{c}{u}_{s0}\\ v_{t0}\\ w_{p0}\end{array}\right)=\left(\begin{array}{c}{r}_{11}{u}_0+r_{21}{v}_0+r_{31}{w}_0\\ r_{12}{u}_0+r_{22}{v}_0+r_{32}{w}_0\\ r_{13}{u}_0+r_{23}{v}_0+r_{33}{w}_0\end{array}\right)=\left(\begin{array}{c}\frac{r_{31}}{\lambda }\\ \frac{r_{32}}{\lambda }\\ \frac{r_{33}}{\lambda }\end{array}\right).$$

(13)

Figure 5. Spectrum shift problem. (a) Spectrum in global coordinate system, (b) spectrum in affine system and (c) spectrum with frequency offsetting.

From the above equation, we can see that the zero frequency in the affine coordinate system has a frequency offset, and this offset will be represented in the reconstruction image with a phase factor. To eliminate this offset, as shown in Figure 5c, we can subtract this frequency offset ${\left(\mathsf{\Delta }{u}_s,\mathsf{\Delta }{v}_t,\mathsf{\Delta }{w}_p\right)}^t$ according to

$$\left(\begin{array}{c}u{\prime }_{s0}\\ v{\prime }_{t0}\\ w{\prime }_{p0}\end{array}\right)=\left(\begin{array}{c}{u}_{s0}\\ v_{t0}\\ w_{p0}\end{array}\right)-\left(\begin{array}{c}\mathsf{\Delta }{u}_s\\ \mathsf{\Delta }{v}_t\\ \mathsf{\Delta }{w}_p\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ \frac{1}{\lambda }\end{array}\right),$$

(14)

where $\Delta u_s=\frac{r_{31}}{\lambda },\Delta v_t=\frac{r_{32}}{\lambda },\Delta w_p=\frac{r_{33}}{\lambda }-\frac{1}{\lambda }.$ Therefore, the spatial frequencies in the “offset” affine coordinate system can be rewritten as

$$\begin{array}{c}u{\prime }_s=r_{11}u+r_{21}v+r_{31}w-\frac{r_{31}}{\lambda }\\ v{\prime }_t=r_{12}u+r_{22}v+r_{32}w-\frac{r_{32}}{\lambda }\\ w{\prime }_p=r_{13}u+r_{23}v+r_{33}w-\frac{1}{\lambda }\left(r_{33}-1\right)\end{array}.$$

(15)

The spectrum of a tilted triangle on the hologram plane now has been completely analyzed, and we can directly obtain the spectrum distribution through the initial parameters. For a single tilted triangle, we have $F_{HOLO}\left(u,v\right)=F_{HOLO,i}\left(u,v\right)$ as in Equation (2), and for a 3D object we use Equation (1). The complex field distribution of a 3D object reconstructed by the hologram can be expressed as the superposition of a tilted triangle complex field on reconstructed image plane $z=z_r$:

$$f_{re}\left(x,y,z_r\right)={ℱ}^{-1}\left\{e^{-j2\pi w{z}_r}·F_{HOLO}\left(u,v\right)\right\},$$

(16)

where $z_r$ is the distance between the reconstructed image plane $z=z_r$ and the hologram plane $z=0$. Through the above steps, we can obtain the reconstructed complex field distribution of the 3D object through only one Fourier transform. In the next section, we will verify our proposed method through numerical simulations and optical experiments.

4. Simulations and Optical Experiment

4.1. Numerical Reconstruction

Based on the 3D mesh in Figure 1, the Stanford bunny consists of 59,996 polygons, and we have reconstructed the bunny using our proposed method. The actual size of the bunny was 3.11 × 3.08 × 2.41 mm³. We have increased its size to 6.56 × 6.49 × 5.08 mm³ before the generation of the hologram. In order to improve the computational efficiency and image quality, we have implemented back-face culling by judging normal. The normal vector of a hologram is $\overrightarrow{n_h}$, and $\overrightarrow{n_u}$ is the normal vector of a tilted triangle. If $\overrightarrow{n_h}·\overrightarrow{n_h}\ge 0$, the tilted triangle will be calculated, and tilted triangles that do not meet the condition will be discarded.

After back-face culling, the bunny only contains 31,724 polygons. However, in Section 3 the result of Equation (12) is based on the assumption that the amplitude distribution of a triangle is a unit constant and the reconstructed results will lack realism. As shown in Figure 6, in order to reproduce the details of the 3D object, we assign the surface function of the tilted triangle $I\left(x,y\right)$ as follows:

$$I\left(x,y\right)=\left[\overrightarrow{n_{ux}},\overrightarrow{n_{uy}},\overrightarrow{n_{uz}}\right]·{\left[cos\alpha ,cos\beta ,cos\gamma \right]}^t+bias=constant,$$

(17)

where $\overrightarrow{n_{ux}},\overrightarrow{n_{uy}},\overrightarrow{n_{uz}}$ are the components of unit normal vector $\overrightarrow{n_u}$ of the xyz-axis in the global coordinate system, and $bias$ represents ambient reflected light. $cos\alpha$, $cos\beta$ and $cos\gamma$ are the direction cosine values of illumination directions. In our case, $bias=0.2,\alpha =60°,\beta =60°,\gamma =5°$.

Figure 6. Principle of flat shading.

The surface function usually refers to the strength information, and the amplitude is expressed as $\sqrt{I\left(x,y\right)}.$ The surface function of the titled triangle $I\left(x,y\right)$ is a constant for each tilted triangle according to Equation (17), so that we can let $I=I\left(x,y\right)$ (I is a constant here) and by using the property of Fourier transform: $\sqrt{I}·f\left(x,y\right)={ℱ}^{-1}\left\{\sqrt{I}·F\left(u,v\right)\right\},$ the spectrum in the hologram of a tilted triangle with added flat shading and the elimination of the “offset” is based on Equation (12), together with Equation (15), we have:

$$F_{HOLO}\left(u,v\right)=\sqrt{I}·J·{\mathrm{e}}^{-j2\pi \left(u{x}_1+v{y}_1+w{z}_1\right)}·F_{\Delta }\left(u_s\prime ,v_t\prime \right).$$

(18)

Figure 7 shows the numerical reconstructions of the bunny based on our proposed F3DAAT method. Figure 7a is the result of calculating Equation (12), and we can see that because the amplitude of each mesh is the same constant the reconstruction result is a lack of realism. Additionally, the bunny has self-occlusion, so that back-face culling by judging normal will have some errors for some polygons. Due to the wrong judgment, visible polygons and invisible polygons are superimposed together and the reconstructed part of the image is shown in the red box on Figure 7a. Figure 7b,c are the reconstruction results based on Equation (18), and from these two reconstructed images we can clearly see the details of various parts of the bunny. Figure 7b focuses on the bunny’s leg, shown in the yellow box, and Figure 7c focuses on the bunny’s ear, shown in the yellow box.

Figure 7. Numerical reconstruction of the bunny. (a) Focus on leg without shading, (b) focus on leg with shading, (c) focus on ear with shading.

We have also scanned a human face called “Alex” with a 3D camera, and the 3D mesh of “Alex” is shown in Figure 8a, which consists of 49,272 meshes. In this case, there is no need to use back-face culling, because the data is the result from actual image scanning. We have calculated the CGH of “Alex”, and its holographic reconstructed image is shown in Figure 8b.

Figure 8. Numerical reconstruction of “Alex”: (a) 3D mesh of Alex and (b) numerical reconstruction. The geometric surface with the texture image in (a) is from a publicly accessible geometric archive from the 3D Scanning Laboratory in Stony Brook University [42].

4.2. Comparison with Previous Methods

The Pseudo Inverse Matrix Method by Pan et al. [25,26] sets up an affine transformation matrix that contains all the information on the transformation. However, they have defined the primitive triangle located at $z_s=0$, i.e., the plane of the source local coordinate system, leading to the concept of a pseudo inverse matrix to perform matrix inversion. The introduction of the pseudo inverse matrix has produced calculation errors and slowed down the calculation speed. Zhang et al. [29] have introduced a Full Analytical 3D Affine Method to avoid the use of the pseudo inverse matrix. The method includes three core steps: rotation transformation for the tilted triangle until it is parallel to the hologram plane, 2D affine transform of the rotated triangle and finally the computation of the field distribution on the hologram by using the angular spectrum (AS) method for diffraction. Zhang et al. [8] also have proposed a Fast 3-D Affine Transformation (F3DAT) method based on the 3D affine transformation by Pan et al., to improve the computation efficiency. In the method, they have defined the primitive triangle located at $z_s\ne 0$, allowing the affine matrix to be fully inverted. The result of the F3DAT method provides a faster calculation time compared with that of the pseudo inverse matrix method and full analytical 3D affine method.

In the present proposed fast 3D analytical affine transformation (F3DAAT) method, we have obtained the affine matrix directly and derived an analytical expression of the spectrum of the primitive triangle. The more meshes that are calculated the more time F3DAAT will save. We have generated the holograms with a resolution of 1024 × 1024, and the hardware includes Intel Core i7-11700 @ 4.8GHz, 16G-byte RAM under the environment of MATLAB 2018b.

The Stanford bunny consisting of 31,724 meshes (after back-face culling) takes 893 s for the calculation, and the calculation of “Alex” of 49,272 meshes takes about 1288 s (See Figure 9). Additionally, shown in Figure 9 where we have used “Alex”, we can see that the computational efficiency of F3DAAT has increased by almost two times compared with the previous methods. The calculation of the four methods is based on the same hardware condition and only CPU is used for the calculation. In one of the most recent studies, Wang et al. [43] proposed a polygon-based method using LUTs (look-up tables) with principal component analysis to speed up the calculation of CGHs. However, the method in the process of pre-computing the affine matrix still needs to solve a pseudo inverse matrix, and our proposed method is more general and efficient for solving the mapping relationship between the two coordinate systems, the global and the affine coordinate systems.

Figure 9. Computation time required for different number of meshes for different methods for “Alex”.

4.3. Optical Experiment

The optical experiment is shown in Figure 10. We have reconstructed 3D objects by loading the phase of the CGHs from the computer onto the spatial light modulator (SLM). The SLM in our experiment is a HOLOEYE PLUTO2 (NIR-011) phase-only SLM with a resolution of 1920 × 1080 (Full HD 1080p) and a pixel pitch of 8 μm and the active area is 15.36 mm × 8.64 mm. The laser is a green light with a wavelength of 532 nm. The spatial filter is used to generate a collimated light for the illumination of the hologram. The polarizer is for adjusting the polarization state of the light to work with the phase-only SLM. A camera (MMRY UC900C Charge-coupled Device) is used to receive the reconstructed image in the image plane of the imaging lens (focal length is 150 mm). Optical reconstructions of the bunny and the face of “Alex” are shown in Figure 11a,b, respectively.

Figure 10. Optical reconstruction experiment.

Figure 11. Results of optical reconstruction: (a) bunny and (b) Alex.

5. Conclusions

In conclusion, we have proposed an improved algorithm to obtain a full-analytical spectrum of a tilted triangle based on 3D affine transformation. Our method avoids the time-consuming steps such as the need to solve for the pseudo inverse matrix or the complex process of 3D rotation and transformation. We have verified our method by calculating complex 3D objects composed of tens of thousands of meshes. In addition, we have added flat shading for realistic image presentation. We have successfully obtained the reconstructed images by numerical and optical reconstructions. Through comparison, it is found that our method improves the computational efficiency by about two times compared with the previous affine methods.