A Sparse Capture Light-Field Coding Algorithm Based on Target Pixel Matching for a Multi-Projector-Type Light-Field Display System

Abstract

The traditional light-field coding algorithm used in a multi-projector-type light-field display system requires sophisticated and complex three-dimensional modeling processes or parallax images obtained through dense capture. Here we propose an algorithm based on target pixel matching, which directly uses parallax images without a complex modeling process, and can achieve a more accurate light-field reconstruction effect under sparse capture conditions. For the lack of capture information caused by sparse capture, this algorithm compares the pixel similarity of the captured images of the same object point on different cameras to accurately determine the real capture information of the object point at different depths, which is recorded as the target pixel, and then the target pixel is encoded according to the lighting path to obtain the correct projector image array (PIA). By comparing the quality of PIAs generated by the traditional light-field coding algorithm and the display effect after loading the PIAs into the actual display system, we proved the effectiveness of the algorithm.

1. Introduction

Light-field display [1] is a new three-dimensional (3D) display technology that gives the observer an immersive stereoscopic visual experience by reconstructing the real light field, which includes integrated imaging technology [2,3,4], grating 3D display technology [5,6,7], holographic 3D display technology [8,9,10], and multi-projector-type 3D display technology [11,12,13,14]. The multi-projector-type 3D display technology has excellent development potential and advantages owing to its easily expanded space bandwidth product. This technology mainly consists of a projector array and a light-regulating device, which projects a large amount of controllable light to reconstruct the expected 3D light field.

Similarly to an integrated imaging system, the multi-projector-type light-field display (MPTLFD) system needs to obtain the encoded projector array image (PIA) through the calculation to control the display ray cast by each projector. For the MPTLFD system, common algorithms can be divided into the light-field coding algorithm based on the virtual 3D model and the light-field coding technology based on parallax images (PIs). Among these, light-field coding based on virtual 3D models is widely used, such as the f-Vision system established by Shunsuke in 2013 [12], and the 360° immersive projector array display system proposed by Liu Xu et al. of Zhejiang University in 2014 [14]. Such coding algorithms first need to model objects, obtain all the depth information of the model in advance, and assign values to display pixels by using the light intensity information at the intersection of the display light and the model. The coding principle is relatively simple, while the innovation work of different teams mainly focuses on the hardware structural design. However, the disadvantage is that the modeling process is more complex. On the other side, the light-field coding algorithm based on PIs is rarely used. In 2010, Balogh et al. [15] directly used the dense sampling camera to capture the parallax image to establish the one-to-one mapping relationship between the display pixel space and the sampling pixel space. This algorithm does not require complex digital modeling. At the same time, the reconstruction effect can retain the true details of the original scene, such as light and shadows, and is more suitable for the light-field display system that reconstructs the real scene. However, under the condition of sparse sampling, owing to the small amount of sampling information, some display rays cannot match their exact corresponding sample, resulting in large errors of the generated projection image array, which affects the reconstruction effect, as shown in the relevant research on integrated imaging [16].

For this reason, this paper proposes a light-field coding algorithm based on target pixel matching. We determine the imaging pixels of the reduction object point required for a display light at the two nearest cameras, and use the pixels to assign a value to the display light, under the condition of sparse sampling; therefore it is still possible to establish a precise mapping relationship between the display pixel and the sampling pixel to generate a high-quality projection image array to improve the light-field reconstruction effect. The effectiveness of the proposed algorithm was verified by experiments.

In Section 2, we briefly introduce the basic structure of the MPTLFD system and the principle of the traditional light-field coding algorithm applied to the system. In Section 3, we present the theory of the algorithm based on target pixel matching. In Section 4, we demonstrate the effectiveness of the algorithm through experiments and the results are also illustrated.

2. Structure of the MPTLFD System and Its Traditional Light-Field Coding Algorithm

2.1. Structure of the Display End and Capture End of the MPTLFD System

A typical structure of a 3D light-field display and capture system based on a projector array is shown in Figure 1. Every projector is uniformly distributed on the circular arc track with the radius $R_P$ according to the interval angle ${\theta }_P$, and the included angle between the optical axis of a projector or camera, and the positive direction of the $x\text{-}axis$ is called the azimuth angle. The azimuth angle range of the projector track is $\left[{\Psi }_{min},{\Psi }_{max}\right]$, and every capture camera is evenly arranged on the radius $R_C$ according to the interval angle ${\theta }_C$, and the azimuth range is $\left[-\Phi ,\Phi \right]$. $R_P$ and $R_C$ can be different, but they are set to be equal here for ease of calculation. The optical axis of each projector and camera coincides at the center of the track circle. A holographic diffuser with anisotropic light modulation is located between the projector track and the camera track, and the center of the diffuser coincides with the track center. The projected light of each projector gathers at the diffuser, and the 3D light field is reconstructed through the scattering of the diffuser. As the human eye is more sensitive to the change of parallax in the horizontal direction than in the vertical direction when viewing directly against the screen to use the limited data bandwidth to obtain a better stereo vision experience, the system discards the parallax in the vertical direction and only restores the light-field distribution in the horizontal direction. The light field with a smooth horizontal motion parallax is reconstructed by using the projector array arranged in horizontal arcs and the diffuser with a small horizontal scattering angle, which gives the observer a better stereo vision experience and produces a 3D effect with a larger field angle, higher resolution, and more delicate display effect in the horizontal direction.

2.2. The Principle of the Improved SPOC Algorithm Applied to a MPTLFD System

As the projector array in the MPTLFD system casts light similarly to the lens array in the integrated imaging (II) system, the smart pseudoscopic-to-orthoscopic conversion (SPOC) [17] algorithm in the II system can also be used for light-field coding in the MPTLFD system. In the SPOC algorithm, the parameters of the lens array designed for the simulated display end are consistent with those of the capture lens array, so the captured parallax images (PIs) can be used as the elementary image array (EIA) of the simulated display end. To solve the depth inversion problem, it directly used the PI at the simulated display end for optical field coding and synthesized the primitive image array corresponding to the parameters of the real display lens array. When the reference plane in the SPOC algorithm is selected consistently with the plane of the EIA, this step can be equivalent to the direct mapping process from the capture pixel space to the display pixel space.

The SPOC algorithm has become mature after a long-period of development, but it still needs to be appropriately modified when it is applied to the MPTLFD system for coding. The corresponding mapping relationship is modified according to the actual light path, and the algorithm is improved to adapt to the capture and reconstruction parameters of the system. The principle of the improved algorithm applied to the system is shown in Figure 2, which is the top view of the system section at the height of the projector array. The projector array and the capture camera array are arranged in concentric arcs and their configuration are shown in Figure 1a. The camera parameters in the capture space and the projector parameters in the projector array are consistent. Line segment $AB$ in Figure 2 represents the top view of the diffuser.

Set

$J\times K$ as the resolution of the capture camera;

${\theta }_{hov}$ as the horizontal field of view (FOV);

$M\times N$ as the resolution of the projector array;

For a pixel $P_n$ projected by a projector $P(x_P,y_P)$, use $l_{P{P}_n}$ represents the display ray between $P$ and $P_n$, and its geometric expression is:

$$y=\frac{y_P-y_n}{x_P}\cdot x+y_n$$

(1)

which is recorded as

$$y=k_d\cdot x+b_d$$

(2)

Combining the orbit equation of the capture camera $x^2+y^2=R_C^2$, we can obtain the coordinate of the ideal camera ($C_{ideal}$) and then calculate its azimuth ${\phi }_{ideal}$. However, owing to the discrete capture of the capture camera, there is often no capture camera exactly corresponding to $C_{ideal}$, in which case the nearest camera is selected; its serial number is $C$, and its azimuth is φ, where:

$$C=round\left[\left({\phi }_{ideal}+\varphi \right)/{\theta }_C\right]$$

(3)

Taking the plane of the diffuser as the reference plane in the SPOC algorithm, we mark the intersection point of the display ray and the reference plane as $S$, where $S$ coincides with $P_n$. The nearest camera ($C$) is used to capture $S$ to approximate the capture of $P_n$ by $C_{ideal}$, which is combined with the FOV of the capture camera (${\theta }_{hov}$) to determine pixel $C_k$ by Equation (4) corresponding to the capture ray between $C$ and $S$ ($l_{CS}$) on the imaging plane ($I$):

$$C_k=round\left\{\frac{K}{2}+\frac{\tan \left(\theta \right)}{\tan \left(\frac{{\theta }_{hov}}{2}\right)/\left(\frac{K}{2}\right)}\right\},\theta =arctan\left(k_d\right)-\phi$$

(4)

where $\theta$ represents the angle between the optical axis of $C$ and the capture ray $l_{CS}$.

In fact, the system only restores the horizontal motion parallax of the object, while in the vertical direction, the divergence angle of the diffuser is large enough to ensure that 3D objects can be observed only in a wide range of vertical field angles, which also causes the mapping relationship of the coding algorithm to remain unchanged at each vertical height. Therefore, the mapping relationship between the capture column pixels and the display column pixels can be established according to Equation (5) to calculate each column of pixels projected by each projector in turn. Finally, the correct projector image array (PIA) is obtained.

$$P_n^m=C_k^m,m\in {\left[0,M-1\right]}_{}$$

(5)

where $P_n^m$ represents the value of the pixel in the $mth$ column and $P_{n}th$ row of the $Pth$ projector image and $C_k^m$ represents the value of the pixel in the $mth$ column and $C_{k}th$ row of the $Cth$ capture image.

In the experiment, we found that when the camera spacing was too large, the quality of the PIA using the SPOC coding algorithm decreased, resulting in a poor light-field reconstruction effect. This is because, as shown in Figure 2, the display ray $l_{P{P}_n}$ is expected to restore the information of object point $O_R$, while in the SPOC algorithm, the nearest camera’s corresponding capture ray is used to replace the ideal camera to assign a value to the display ray, and the capture ray actually captures the light-field information at object point $O$, thus introducing an error. At the same time, the error increases with the increase of the capture camera’s arrangement interval.

3. Sparse Light-Field Coding Algorithm Based on Target Pixel Matching

To solve the error in the SPOC algorithm under sparse capture conditions, this paper proposes an improved light-field coding algorithm based on target pixel matching. Considering that the difference of the same object point’s light intensity captured by adjacent cameras is small, by searching for the pair of most similar pixel columns in the two cameras nearest to the display ray, the algorithm can obtain the correct imaging pixel columns of the actual object, which are then used to assign values to the display ray. This algorithm can establish a more accurate mapping relationship between the capture pixel and the display pixel under the condition of low capture density, synthesize a high-quality PIA, and significantly improve the light-field reconstruction effect.

As shown in Figure 3, the optical center of the two nearest capture cameras $C_A,C_B$ can be obtained by Equation (6), and obtains the camera azimuth ${\phi }_A,{\phi }_B$, where:

$$C_A=\lfloor \frac{{\phi }_{idedal}+\varphi }{{\theta }_C}\rfloor ,{\mathrm{C}}_B=C_A+1$$

(6)

the object point $O_R$ expected to be restored for the display ray $l_{P{P}_n}$ is located on the line segment between the point $P$ and the point $C_{ideal}$. Record the imaging pixels of $O_R$ captured by $C_A$ and $C_B$ as $I_{O_A},I_{O_B}$, and they can be, respectively, obtained by calculating the intersection of the imaging plane and the capture ray from $O_R$ to the optical center of the camera. In other words, $O_R$ is located at the intersection of the display ray and the capture ray from the optical center of the camera to the imaging pixel of $O_R$. Therefore, this algorithm first determines the interval where $I_{O_A}$ exists in $C_A$, for any pixel within the interval, a capture ray can be determined from the pixel to the optical center of the camera. As mentioned above, $O_R$ is located at the intersection of the display light and a certain capture ray, according to the intersection, the corresponding imaging pixel in $C_B$ can be determined, and multiple pixel pairs can be established by traversing all pixels within the interval. Generally, an object can be regarded as a Lambert body, and the light emitted by the same object is isotropic at all angles. In fact, the distance between the object and the capture camera is far greater than the optical center interval of the camera, when the adjacent cameras capture the same object point, the included angle of the capture rays is small, and the difference of the capture rays intensity information is small, the values of the imaging pixels camera are similar, so, $I_{O_A}$ and $I_{O_B}$ are the group of pixel pairs with the highest similarity. The schematic diagram of the algorithm is shown in Figure 4 and Figure 5.

To reduce the computational redundancy, the algorithm first establishes a virtual activity space, which is a cylinder space with a sufficiently large radius, and the space can contain all of the light-field information of the real scene. As shown in Figure 4, this figure is the top view of the system structure at the height of the projector array, the blue area with a radius $R_A$ is the virtual activity space. $l_{P{P}_n}$ intersects the boundary of the active space at two points, which are recorded as the near field point, $Near\left(x_N,y_N\right)$, and far field point, $Far\left(x_F,y_F\right)$ according to the distance between these two points and the camera’s orbit.

According to the relationship between the far and near field points and the optical center $C_A$, and in combination with the FOV of the camera, the algorithm calculates the imaging pixels $I_{F_A},I_{N_A}$, where:

$$I_{F_A}=round\left\{\frac{K}{2}+\frac{\tan \left({\theta }_{F{C}_A}\right)}{\tan \left(\frac{{\theta }_{hov}}{2}\right)/\left(\frac{K}{2}\right)}\right\},{\theta }_{F{C}_A}={\phi }_{F{C}_A}-{\phi }_A$$

(7)

${\theta }_{F{C}_A},{\phi }_{F{C}_A}$ represent the angle between the capture ray $l_{F{C}_A}$ and $C_A$ and the $x\text{-}axis$, respectively; then $I_{N_A}$ can be obtained in the same way.

The real object point $(O_R)$ to be restored for $l_{P{P}_n}$ is located in the area of the line segment $\left(l_{FA}\right)$ between $Near$ and $Far$, so $I_{O_A}$ meets the following relationship:

$$0\le I_{N_A}\le I_{O_A}\le I_{F_A}\le K-1$$

(8)

Equation (8) determines the interval where the imaging pixel of $O_R$ exists and then determines the actual imaging pixel of $O_R$. Assume that the imaging pixel of $O_R$ in camera $C_A$ is $I_{S_A}$, and $I_{S_A}\in \left[I_{N_A},I_{F_A}\right]$. Then, the angle between the capture ray and the $x\text{-}axis$ can be deduced by Equation (7), and then the mathematical expression can be obtained, which is recorded as:

$$y=k_C\cdot x+b_C$$

(9)

The coordinate of intersection $P_{oI}$ of the capture ray from $I_{S_A}$ to $C_A$$(l_{I_{S_A}{C}_A})$ and $l_{P{P}_n}$ is as follows:

$$P_{o_I}\left(x_{P_{o_I}},y_{P_{o_I}}\right),\{\begin{array}{l}{x}_{P_{o_I}}=(k_C-k_d)/(b_d-b_C)\\ y_{P_{o_I}}=k_C\times x_{P_{o_I}}+b_C\end{array}$$

(10)

Then, $O_R$ is at $P_{oI}$. For the adjacent camera $C_B$, the imaging pixel of $O_R$ is the intersection point of the imaging plane and the capture ray from $P_{oI}$ to $C_B$($l_{P_{{\mathrm{o}}_I}{C}_B}$), and then the imaging pixel is recorded as $I_{S_B}$:

$$I_{S_B}=round\left\{\frac{K}{2}+\frac{\tan \left({\theta }_O\right)}{\left[\tan \left(\frac{{\theta }_{hov}}{2}\right)/\left(\frac{K}{2}\right)\right]}\right\},{\theta }_O={\phi }_O-{\phi }_B$$

(11)

where ${\theta }_O,{\phi }_O$ represent the angle between the capture ray $l_{P_{{\mathrm{o}}_I}{C}_B}$ and the optical axis of $C_B$ and the $x-axis$, respectively. According to the geometric relationship, multiple pixel pairs are obtained by traversing all pixels within the interval $\left[I_{N_A},I_{F_A}\right]$.

As shown in Figure 5b, when $I_{S_A}\ne I_{O_A}$, capture rays $l_{I_{S_A}{C}_A}$, $l_{P_{{\mathrm{o}}_I}{C}_B}$ and the real object surface intersect at $O_A$, $O_B$, respectively, it actually represents that $I_{S_A}$ is the imaging pixel of $O_A$ by camera $C_A$, while $I_{S_B}$ is the imaging pixel of $O_B$ by camera $C_B$. $O_A$ is completely unrelated to $O_B$, so the imaging pixels are different. When the assumed imaging pixel $I_{S_A}$ is the real imaging pixel $I_{O_A}$, $I_{O_A}$ and $I_{O_B}$ represent the imaging pixels of the same real object point $O_R$ in the two nearest cameras, and the pixel values are similar at this time. To simplify the calculation, we use the variance value to represent the similarity of the pixels, and the smaller the variance, the more similar the pixels. According to the above analysis, the imaging pixels of the real object point in the two nearest cameras can be given by traversing the pixel pairs with geometric correspondence and obtaining the pixel pair with the smallest variance. Similarly, as mentioned above, the mapping relationship of the light-field coding algorithm does not change at each vertical height, so the object point information $O_R$ actually represents a continuous list of object point information, of which the light field is captured dispersed into $J$ pixels, where $J$ represents the camera’s vertical resolution, so the imaging pixel columns of the real object point $O_R$ at the two nearest cameras can be calculated by Equation (12) when $s^2={\displaystyle \sum _{j=0}^{J-1}\frac{{(I_{S_A}^j-(I_{S_A}^j+I_{S_B}^j)/2)}^2+{(I_{S_B}^j-(I_{S_A}^j+I_{S_B}^j)/2)}^2}{2}}$ is minimum.

$$I_{O_A}=I_{S_A},I_{O_B}=I_{S_B}$$

(12)

Here, $I_{S_A}^j$ and $I_{S_B}^j$, respectively, represent the value of the $jth$ row of column pixel $I_{S_A}$ in the $C_{A}th$ captured image and the value of the $jth$ row of column pixel $I_{S_B}$ in the $C_{B}th$ captured image.

The light emitted from the same object point to different angles has a correlation. The smaller the angle between the lights, the greater the correlation. In the process of pixel synthesis, the smaller the angle between the capture ray and the display ray, the greater the impact of the capture pixel on the display pixel. Therefore, the capture pixel is weighted and fused into the pixel value of the PIA by interpolation according to the mapping relationship in Equation (13). Calculate each column of pixels projected by each projector in turn, and finally obtain the correct PIA.

$$P_n^m=\Delta \cdot I_{O_A}^m+\left(1-\Delta \right)\cdot I_{O_B}^m,\Delta ={\theta }_B/\left({\theta }_A+{\theta }_B\right)$$

(13)

where ∆ is the weight; ${\theta }_A,{\theta }_B$ are the angles between the two capture rays and the display ray $l_{P{P}_n}$; $P_n^m$, $I_{O_A}^m$, $I_{O_B}^m$ represent the pixel value of the $mth$ row pixel in the display pixel column $P_n$ and capture pixel column $I_{O_A},I_{O_B}$, respectively.

This approach can be summarized as follows:

(1)

Calculate the two capture cameras that are closest to a display ray cast by the projector $P$;

(2)

Determine the interval of the imaging pixel of the object that the display ray needs to restore;

(3)

Determine the coordinate of the intersection of the display ray and the line between a pixel in the interval and the optical center of the camera;

(4)

Calculate the image pixel of the intersection coordinate taken by another nearest camera;

(5)

Traverse all pixels in the interval to obtain multiple pixel pairs with the above relation;

(6)

Find the pixel pair that minimizes $s^2$ to determine the imaging pixels of the object;

(7)

Calculate the display pixels values by Equation (13).

4. Experimental Verification and Analysis

The display system used in the experiment consisted of 108 projectors with a resolution of 1280 ∗ 720 and a holographic diffuser. The projectors were uniformly distributed on the circular track with a radius of 2.14 m and an azimuth of [153°, 207°] according to an interval angle of 0.5°. The center of the diffuser coincided with the center of the projector track and was perpendicular to the optical axis of the central projector (with an azimuth angle of 180°). The specific display system structure is shown in Figure 6.

4.1. Structure of the Virtual Scene Capture End

To verify the effectiveness of the algorithm, first, we selected the 3DMax software to capture the 3D light-field information. The structure and scene settings of this capture system are shown in Figure 7a. The FOV of the virtual camera was28°, and the capture image resolution was1280 ∗ 720. The cameras were evenly distributed on the circular arc track, with a radius of 2.14 m and an azimuth of [−48.6°, 48.6°], according to a 1.8° spacing angle. There were 55 cameras in total, and the optical axes of each camera converged at the center of the track. The captured object was a teapot with a cartoon sticker. The geometric center of the teapot coincided with the center of the camera track.

4.2. Experimental Results and Analysis

Following capturing the sparse PIs by the above method, we compare the PIA generated by the proposed algorithm with the PIA generated by the SPOC algorithm, and the results are shown in Figure 8. On the whole, the two PIAs are similar in terms of image structure, teapot contour, cartoon sticker patterns, and the distribution patterns. For the shadow areas caused by uneven lighting, the two images have similar rendering effects. However, the details are different. The quality of the PIA is better when using the target pixel matching algorithm, and the layout of the whole image in Figure 8b is fine and smooth, without the severe stripe mosaic phenomenon shown in Figure 8a. Compared with the original PIs, such as the eyeball of a cartoon sticker, the lines shown in Figure 8a are disordered, while those in Figure 8b are even, which is close to the original scene. For the part with complex depth information in the original scene, the spout in Figure 8a has a severe ghosting phenomenon, and it is difficult to distinguish the depth change relationship between the spout and the body. The spout in Figure 8b is smooth and round, and there is a distinct boundary between the spout and the body. This is because in the SPOC algorithm, it is difficult to select a single reference plane to reflect the complex depth information of the continuous changes of objects. In this algorithm, the reference plane is no longer introduced manually, and the similarity comparison of target pixels is used to obtain the imaging pixels of real objects and assign values to the PIA. Under the sparse capture condition, the imaging information of objects at different depths can be restored more accurately.

The peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), which are commonly used in image quality evaluation, are introduced to test the performance of the proposed algorithm. The PIA generated by capturing PIs under dense capture conditions was used as the base image, and the PIAs generated by the two algorithms under the condition of sparse capture as experimental images. PSNR and SSIM were calculated in the following two cases:

case 1: compare the first single projection image of the base image with the first single projection image of the experimental images;

case 2: compare all the projection images of the base image with all of the projection images of the experimental images.

The results are shown in the

Table 1. Comparison of PSNR and SSIM of the two algorithms in case 1.

Algorithm	PSNR(dB)	SSIM	Encoding Time(S)
SPOC	38.68	0.65	15.69
Target pixel matching	39.93	0.82	120.24

and

Table 2. Comparison of PSNR and SSIM of the two algorithms in case 2.

Algorithm	PSNR(dB)	SSIM	Encoding Time(S)
SPOC	39.13	0.72	15.69
Target pixel matching	40.76	0.88	120.24

, which also compare the encoding time of the two algorithms (the time of calculating a single projection image from the PIs through the optical field-coding algorithm). For whether a single projection image or all PIAs, the PSNR and SSIM values of the images generated by this algorithm are higher, a higher PSNR means a smaller distortion rate compared with the base image, a higher SSIM indicates that the experimental image has a higher similarity with the base image in contrast, brightness, structure and other aspects. In short, PIA generated by this algorithm are more similar to the original scene, proving the effectiveness of the proposed light-field encoding algorithm.

However, the encoding time of the proposed algorithm is longer, the algorithm speed is mainly limited by the number and resolution of projectors and the selection of the radius of the virtual activity space. When the projector interval is increased or its horizontal resolution is reduced, the algorithm encoding time is also reduced accordingly, but in the display effect, the motion parallax in the horizontal direction is not smooth enough. When the radius of the virtual activity space is reduced, because the distance between the far and near field points is reduced, the determined imaging interval in a camera becomes smaller, and the time cost is also reduced when searching for the most similar pixel pairs by traversing all possible imaging pixels in the imaging interval. However, when the virtual activity space is too small to contain all of the light-field information of the object, all of the information of the original 3D scene cannot be reconstructed. In the experiment, affected by the system structure, the PIA was synthesized pixel column by pixel column. The encoding process of each column of display pixels is independent and irrelevant, so the algorithm speed can be improved through a parallel operation.

We loaded the PIAs encoded by the SPOC algorithm and the algorithm proposed in this paper into the MPTLFD system to observe the actual display effect, and the comparison of the display results is shown in Figure 9. It can be seen that the SPOC algorithm has a poor reconstruction effect, severe color aliasing, fuzzy cartoon stickers, and an irregular arrangement of stripe mosaics. On the contrary, in the reconstructed light-field effect based on the target pixel matching algorithm, first, as the perspective changes, the reconstructed object shows smooth and delicate different sides and has complete and continuous horizontal motion parallax and a good stereo sense. In addition, in the SPOC algorithm, in the left 30° view, it is obvious that the right side of the reconstructed object has a phantom beyond the contour of the teapot; in the middle view, the spout and the body overlap, making them almost indistinguishable, and the phenomenon of ghosting is serious. In contrast, in the algorithm proposed in this paper, the contour of the teapot is regular, the boundary between the teapot spout and the teapot body is clear, and the depth information is different, which is more similar to the original scene. This is because for each display ray observed from a certain perspective, it restores different object point information. Owing to the lack of capture information caused by sparse capture, the SPOC algorithm directly uses the nearest camera instead of the ideal camera to assign values to the display ray. As shown in Figure 2, the display ray $l_{P{P}_n}$ that needs to restore the light-field information of $O$ restores the information of $O_I$ by error, the imprecise mapping relationship causes numerous display rays to carry the wrong and the same object information. From the perspective of the observer, they can observe the ghosting phenomenon. At the same time, for the teapot spout, a single reference plane introduced in the SPOC algorithm cannot accurately represent its complex depth information, resulting in no obvious change in the depth of the spout and the body in the final display effect. In this algorithm, according to the target pixel, which is the actual imaging pixel of the object point information that each display ray needs to restore, the display ray is assigned a value to ensure that each it can restore accurate object point information, through this way, this algorithm can effectively solve the ghosting phenomenon, accurately restore the information at different depths, and improve the 3D effect.

Through the above experiments, we confirmed that the proposed algorithm could accurately synthesize the PIA under sparse capture conditions, effectively improve the effectiveness of the light-field reconstruction for complex scenes with depth information, and is more suitable for high-quality light-field display systems.

5. Conclusions

In this paper, a new light-field coding algorithm for synthesizing a PIA was proposed. Under the condition of sparse capture, the PIs obtained by the capture is directly used. First, the auxiliary virtual activity space is set to determine the interval where the imaging pixel of the object exists, and the imaging pixel of the object under different depth information can be accurately determined by finding the most similar pixel pair in the two nearest cameras. The imaging pixel is used to assign a value to the display pixel to establish a more accurate mapping relationship between the capture pixel space and the display pixel space, thereby improving the image synthesis quality. Compared with the model-based light-field coding algorithm, this algorithm does not need a complex modeling process and is more concise and efficient. Compared with the SPOC algorithm, this algorithm does not need to introduce a reference plane, which avoids the problem of introducing errors by using a single reference plane to reflect the continuous changes in the depth information of the actual 3D scene. Through experimental comparison, we found that the PIA synthesized by the algorithm proposed in this paper was clearer and more accurate, and the reconstructed light field was more similar to the original scene after being loaded into the display system. However, this algorithm also has some problems, such as a long encoding time. At the same time, in the face of the problem of light-field reconstruction of the real scene, in addition to the problem of sparse camera arrangement, the camera attitude should also be considered. Subsequent work will focus on the acceleration of the light-field encoding algorithm and the coding of free cameras in a real scene.