Improved Ternary Gray-Code Phase Unwrapping Algorithm for 3D Measurement Using a Binary Defocusing Technique

Abstract

By using one-bit binary patterns instead of eight-bit sinusoidal ones, the binary defocusing techniques have been widely applied for high-speed 3D shape measurement. As projector defocusing is required, the phase unwrapping process of these techniques remains challenging. A recently proposed ternary Gray-code method can effectively increase the measurement speed by reducing the number of acquired coding patterns. However, it still has limitations, including the measuring range and a noise problem. To improve these, a new ternary Gray-code method is proposed by utilizing 2D modulation instead of the 1D modulation used in the conventional method. Simulations were conducted to compare these two methods, and the results verify that the proposed method reduces the variance in the introduced intermediate gray value by nearly 90%. Experiments were also carried out with the results verifying the superiority of the proposed method.

1. Introduction

Three-dimensional (3D) shape measurement has become more and more important in areas ranging from manufacturing to medicine. Numerous techniques, including the Moiré, Stereo vision, and fringe projection techniques, have been developed for different applications. Among these methods, the digital fringe projection technique has been exhaustively studied and widely applied due to its simple setup, automatic data processing, high speed, and high-resolution measurement capabilities [1,2,3].

For high-speed and high-accuracy 3D shape measurement, it has been demonstrated that the use of one-bit binary patterns provides advantages over the use of eight-bit sinusoidal phase-shifted fringe patterns, especially on the digital-light-processing (DLP) projection platform [4].By properly defocusing the projector for specific square binary patterns, researchers have achieved speed breakthroughs for high-accuracy 3D shape measurement [5,6].For the binary defocusing technique, a phase-shifting algorithm or the Fourier transform method is generally adopted to extract the phase information, which is wrapped in the range of (−π, π] with 2π phase jumps. In order to obtain the absolute phase, a phase-unwrapping process is usually required. These algorithms can be classified into spatial and temporal categories [7,8,9]. Spatial phase unwrapping is more straightforward, with the requirement of only one wrapped phase map, but it may fail when phase discontinuities or serious phase noises exist. In contrast, temporal phase unwrapping is considered to be more robust as it uses additional projected patterns; methods that use this technique include the multifrequency [7], phase coding [10,11,12], and Gray-code methods [12,13].

The multifrequency method utilizes multiple sets of phase-shifting fringes with different frequencies or periods for absolute phase calculation. Therefore, it raises the challenge of generating binary fringe patterns with wide periods. The phase coding method uses phase information for codeword encoding; thus, it is relatively robust to ambient light and noise. However, it requires at least two sets of phase coding patterns to reduce the phase errors. The Gray-code method encodes the fringe order into the binary Gray-code patterns, and it has the flexibility to encode large codewords by a proper design. For example, for the conventional Gray-code method, if there are f periods in the projected pattern, then ⌈${\log }_{2}f$⌉ patterns are needed for the Gray code, where $\lceil •\rceil$ is the ceiling function. Recently, a ternary Gray-code method was proposed to reduce the number of coded patterns [14], thus achieving even faster 3D measurement. By using a new three-level pattern design strategy, an additional gray level is realized and the number of required coding patterns can be reduced to ⌈${\log }_{3}f$⌉.

Although it provides faster measurement, the ternary Gray-code method still has some limitations. In order to correctly detect the Gray code, the patterns need to be carefully designed and the threshold values need to be well picked. In addition, the defocusing level also affects the codeword detection, thus limiting the measurement range. To overcome these problems, in this paper, an improved ternary Gray-code method is proposed. Instead of only using the 1D design strategy, 2D modulation is utilized in the proposed method to make sure the generated additional gray level is more uniform; thus, it is more robust to defocusing effect and noises.

The rest of the paper is organized as follows. Section 2 presents the principle of the proposed method. Section 3 provides the experimental results. Section 4 concludes this paper.

2. Materials and Methods

2.1. Phase-Shifting Algorithm

Phase-shifting methods are widely used in optical metrology because of their speed and accuracy [15]. If a multi-step phase-shifting algorithm with a phase shift of δ_n is adopted, the intensity distributions for the fringes can be described as

$$I_n\left(x,y\right)=A\left(x,y\right)+B\left(x,y\right)\cos \left(\mathsf{\Phi }+{\delta }_n\right),n=1,2,3,\dots ,N$$

(1)

where A(x, y) is the average intensity, B(x, y) is the intensity modulation, and Φ(x, y) is the phase to be solved for. The wrapped phase can be calculated by using a least-square algorithm [14].

$$\mathsf{\Phi }\left(x, y\right)=-{\tan }^{-1}\left[\frac{{{\displaystyle \sum }}_{n=1}^N{I}_n\left(x,y\right)\sin {\delta }_n}{{{\displaystyle \sum }}_{n=1}^N{I}_n\left(x,y\right)\cos {\delta }_n}\right]$$

(2)

For binary defocusing techniques, the sinusoidal patterns can be approximated by defocusing different binary structures. When the fringe period T is small, the square binary structure is generally adopted, in which half of the period is black while the other half is white.

2.2. Conventional Ternary Gray-Code Method

To improve the performance of the original Gray-code method for the binary defocusing technique [16], Zheng et al. proposed to generate an additional gray level using black and white values, as illustrated in Figure 1 [14]. Instead of merely using black ($\overline{AB}$) and white ($\overline{EF}$) regions, an intermediate region ($\overline{CD}$) is elegantly designed by using 1 and 0 alternately [14].

To determine the threshold for segmenting the different regions, a 1D Gaussian filter was adopted for analyzing the gray values for the intersection points under different defocusing levels. As demonstrated in [14], by using the Euler–Maclaurin formula, the intensity values at point B, C, D, and E in the conventional ternary coding method are

$$I^b\left(B\right)=\frac{1}{\sqrt{2\pi }\delta }{\displaystyle \sum }_{i=1}^{\infty }{e}^{-{\left(2i-1\right)}^2/2{\delta }^2}=1/4$$

(3)

$$I^b\left(C\right)=1/4+1/\left(2\sqrt{2\pi }\delta \right)$$

(4)

$$I^b\left(D\right)=3/4-1/\left(2\sqrt{2\pi }\delta \right)$$

(5)

I^b(E) = 3/4

(6)

where the standard deviation of the Gaussian filter δ > $\sqrt{2}$. This means that, for the conventional ternary coding method, the intensity value of the edge point B in the black region is already about 1/4, which means that there is an offset around the intersection part.

The intensity difference between B and C, and between D and E, in Figure 1 is

$$\tau =I^b(C)-I^b(B)=I^b(E)-I^b(D)=1/(2\sqrt{2\pi }\delta )$$

(7)

Therefore, the threshold Th₁ for ($\overline{AB}$) and ($\overline{CD}$) to segment the blurred patterns can be expressed as

$$T{h}_1=1/4+\tau /2=1/4+1/(4\sqrt{2\pi }\delta )$$

(8)

The threshold Th₂ for ($\overline{CD}$) and ($\overline{EF}$) is

$$T{h}_2=3/4-\tau /2=3/4-1/(4\sqrt{2\pi }\delta )$$

(9)

According to [14], to obtain a high-fringe contrast, the upper bound of δ should be 3. On this basis, the thresholds can be calculated, which are about Th₁ = 0.28 in Equation (8) and Th₂ = 0.72 in Equation (9) when δ = 3.

Since three gray levels can be generated, the (3, m)-Gray-code mechanism can be applied for fringe order detection with m binary patterns. The Gray code (3, m) provides the following codeword, which is unique for each period.

$$C\left(x,y\right)={\displaystyle \sum }_{k=1}^m\left[S_k\left(x,y\right)\times 3^{k-1}\right]$$

(10)

This can be further converted to a unique fringe order K, with which the absolute phase map can be finally retrieved as

$$\mathsf{\Phi }\left(x,y\right)=\phi \left(x,y\right)+K\left(x,y\right)\times 2\pi$$

(11)

2.3. Improved Ternary Gray-Code Method

The conventional ternary Gray-code method has been proven to be effective at reducing the required number of coding patterns. However, since it only utilizes 1D modulation, the method may have limitations considering defocusing effects and the noise problem. To improve its robustness, in this paper we propose to use 2D modulation to generate the intermediate gray level. The basic principle is shown in Figure 2. To clearly demonstrate the difference, 2D patterns containing three gray levels (i.e., black, gray, and white) for the conventional and improved ternary Gray-code methods are respectively shown in Figure 2a,b.

Changing the modulation from 1D to 2D will bring about significant improvements. First, the introduced intermediate gray value will be more uniform with much smaller variance. As a result, this method is more robust when noises exist. In addition, the performance of 2D modulation will be less sensitive to the defocusing levels, and the working range of the improved method could be extended. In addition, the change in the intersection points will be symmetric and sharp, which may facilitate the segmentation for different regions.

We compare the conventional and improved coding methods in these respects. First, the uniformness for the introduced intermediate gray level is carefully examined with different defocusing levels. The defocusing effect is simulated by applying Gaussian filters with different sizes. The intermediate gray-level patterns were designed according to the two methods as shown in Figure 3. The first row shows the designed binary patterns and, after applying a Gaussian filter with a small size of 3 × 3 pixels, the defocused patterns are shown in the second row, from which it can be seen that there is obvious variance for the conventional method while the result from the improved method is much more uniform.

To better view the difference, the cross sections for the defocused patterns are shown in Figure 4a, which clearly reveal that the improved ternary coding method reduced the variance by close to 90%. In addition, we applied n × n pixel Gaussian filters with a standard deviation of n/3, here n = 3, 5, 7, …, 15, 17, 19. These filter sizes represent various defocusing levels. We then computed the root-mean-square (rms) error for the various defocusing levels with each filter size and plotted the rms error as a function of filter size as shown in Figure 4b. It can be seen that the rms error for the improved method is much smaller than the conventional one under all defocusing levels, which means the proposed method can enhance the 3D measuring range.

Another aspect that needs to be examined is the intersection part between different gray-level regions, such as the sections of ($\overline{BC}$) and ($\overline{DE}$) shown in Figure 1. For the conventional ternary coding method, the thresholds need to be carefully chosen because of the set δ value, which are about Th₁ = 0.28 and Th₂ = 0.72 [14]. Actually, due to the asymmetric character, well-picked thresholds may even fail when the projector is not well defocused. Gaussian filters with different sizes were applied to the designed coding patterns, and the cross sections of the intersection part between the black and gray regions for the conventional and improved methods are plotted as shown in Figure 5a,b. From these figures, it can be seen that it is hard to give an accurate threshold for all defocusing levels for the conventional method since the gray values diverge around the intersection point (10.5 pixels). In contrast, for the improved method the gray values converge and it is easy to set δ = 0; then, Th₁ = 0.25 in Equation (8). Similarly, the other threshold can be accurately set as Th₂ = 0.75 in Equation (9).

3. Experiments

To verify the performance of the proposed ternary Gray-code method, experiments were carried out by measuring complex objects. A 3D shape measurement system was developed including a digital-light-processing (DLP) projector (Model: Light-Crafter 4500) and a digital CCD camera (Model: Point-Grey GS3-U3-50S5M). The projector’s resolution is 912 × 1140 pixels, while the camera’s resolution is 2448 × 2048 pixels with a maximum frame rate of 15 frames/s. The camera uses a 16 mm focal length Mega-pixel lens (Model: Computer M1614-MP). A five-step phase-shifting algorithm was adopted, and five squared binary patterns were generated according to the fringe period of T = 20 pixels.

In the experiments, in order to cover the whole projection range, there should be four coding patterns for both ternary Gray-code methods. Figure 6a–d show the four coding patterns for the original ternary coding method when the projector is nearly focused, while Figure 6e–h show the coding patterns for the proposed method. From the figures, it can be seen that the structures or variations are clearly noticeable for the original method in the intermediate gray region, while the proposed method provides a more smooth intensity distribution.

Such variations could introduce errors when recovering the codewords or fringe order information. From Figure 6, it can be found that for the third coding pattern the variations around the forehead region are more serious. To better view the effects, Figure 7 shows the codeword detection results of the middle cross section. Based on the projected shifts in patterns, the ternary coding patterns can be normalized and distributed.

The middle cross section is shown in Figure 7a. After normalization, it can be seen that the improved method’s performance has smaller variations; thus, it is better than that of the conventional method. In particular, in the forehead region the amplitude of the variations for the conventional method is too large and the gray values are already outside the range for the intermediate gray level. By using the thresholds Th₁ = 0.28 and Th₂ = 0.72 for the conventional method [14], the codewords can be calculated as shown in Figure 7b. The result of the improved method was also obtained with the thresholds Th₁ = 0.25 and Th₂ = 0.75. From Figure 7b, we can notice there are serious codeword errors around the forehead region for the conventional method, while both methods provide good results for other regions.

To better compare the performance, experiments with different defocusing levels were also carried out and the final 3D results are shown in Figure 8. Figure 8a,d respectively show the results of the conventional and the proposed methods when the projector is nearly focused. Figure 8b,e show the results when the projector is slightly defocused, while Figure 8c,f show the results when the projector is significantly defocused. For the conventional ternary Gray-code method, there are obvious errors around the forehead region when the projector is nearly focused and, by increasing the defocusing effect, the number of errors is reduced for the forehead region, which is reasonable since the intermediate gray region would be smoother. However, the significant defocusing effect could introduce other problems, as shown in Figure 8c, because the intersection region will be blurred and there will be shifts in the detected codewords around the boundary regions. For the proposed ternary Gray-code method, the best result is obtained when the projector is nearly focused as shown in Figure 8d. When the projector is slightly defocused, the recovered 3D result still has good quality. However, when the projector is significantly defocused, boundary errors are introduced as shown in Figure 8f.

4. Discussion and Conclusions

In this paper, an improved ternary Gray-code method is proposed to generate an intermediate gray level by utilizing 2D modulation. Compared with the conventional ternary Gray-code method, the proposed method can ensure that the intermediate gray region is more uniform under different defocusing levels; thus, it is more robust to noise. In addition, the threshold values are consistent and can be accurately set for different defocusing levels. Simulations and experiments were carried out to compare the performance of the two methods, and the results verify the improvements of the proposed method.

In system subject to real noise, a large number of fringes, such as binary-coded or ternary-coded fringes, is required as discussed in the work. Deep learning, as an AI technique that has been widely used for image-based tasks, can reduce the number of required fringes [17]. The combination of the proposed method with Deep learning might be able to reduce the number of required fringes while preserving accuracy.