Extending the Depth of Field beyond Geometrical Imaging Limitations Using Phase Noise as a Focus Measure in Multiwavelength Digital Holography

Featured Application

Multiwavelength digital holography on objects beyond the unambiguous measurement range.

Abstract

Digital holography is a well-established technology for optical quality control in industrial applications. Two common challenges in digital holographic measurement tasks are the ambiguity at phase steps and the limited depth of focus. With multiwavelength holography, multiple artificial wavelengths are used to extend the sensor’s measurement range up to several millimeters, allowing measurements on rough surfaces. To further extend the unambiguous range, additional highly stabilized and increasingly expensive laser sources can be used. Besides that, unwrapping algorithms can be used to overcome phase ambiguities—but these require continuous objects. With the unique feature of numerical refocusing, digital holography allows the numerical generation of an all-in-focus unambiguous image. We present a shape-from-focus algorithm that allows the extension of the depth of field beyond geometrical imaging limitations and yields unambiguous height information, even across discontinuities. Phase noise is used as a focus criterion and to generate a focus index map. The algorithm’s performance is demonstrated at a gear flank with steep slopes and a step sample with discontinuities far beyond the system’s geometrical limit. The benefit of this method on axially extended objects is discussed.

1. Introduction

Quickly measuring three-dimensional (3D) surfaces with small 3D structures is a challenge for industrial measurement technology. For 100% inline-inspection of components, the measuring system has to keep up with the production cycle. Because of its great resolution and high speed, digital holography is a well-established technology for nondestructive testing in industrial applications [1,2,3,4,5,6,7]. Millions of data points can be acquired in a fraction of a second, which allows measuring cycles of multiple Hz.

Lateral resolution of holographic sensors is limited by the pixel pitch of the imaging device and designed magnification. With the use of a lens, it can be both improved and adapted. With respect to axial resolution and unambiguity, the wavelength of the laser and the dynamic range of the imaging device are limiting factors. In addition, limited depth-of-focus is a challenge addressed by our algorithm. Spatial unwrapping allows to extend the measuring range, but often fails (e.g., at steep edges and deep holes) [8]. Using digital multiwavelength holography, speckle fields are captured at identical boundary conditions but at different wavelengths. This allows a numerical generation of phase maps of an artificial wavelength. These generated synthetic wavefronts overcome both restrictions [9,10,11,12,13,14,15,16]. As an alternative to the additional expensive lasers and further appropriate longer synthetic wavelengths, the axial measurement range in digital holography can be extended using shape-from-focus algorithms for focus detection, applied on amplitude or phase images. Promising approaches have been highly investigated in the past (e.g., amplitude stack [17], complex ratio [18], correlation coefficient [19], energy conservation [19,20], entropy [19], Fourier spectral [19,20,21,22], Gini index [23], gradient [22], Kirchhoff [24], L1 norm [23,25], Laplacian [21,22,26], multispectral images [19,27], self-entropy [28], speckle phase decorrelation [29], Tamura coefficient [30,31] variance [19,21,26,32,33] Klicken oder tippen Sie hier, um Text einzugeben, quadratic deformation of spatial coordinates [34], cubic phase plate [35], amplitude modulus [36], or numerical axicon transformation [37]).

We present a shape-from-focus algorithm based on phase noise and use its output as a rough estimate of the object’s surface shape, which allows us to extend the measurement range far beyond the limits of synthetic wavelengths and geometrical depth-of-focus. This algorithm has been previously introduced by us in 2017 [38], using a maximum filter in combination with a plane fit for height map generation. In this paper, we present a generalized improvement using a pixel wise Gaussian fit.

The performance of our method is demonstrated using a gear test object at steep slopes and a step object with gaps beyond the measuring unambiguous range. Even at steep slopes, we are able to numerically compute an all-in-focus image of the gear flank beyond the measurement range of the synthetic wavelengths being used. In order to save computing time, data processing is mostly executed on the graphics processing unit (GPU).

2. Experimental Setup

Figure 1a shows an experimental setup for multiwavelength digital holography with an exemplary gear tooth. However, the algorithm introduced in this paper is not limited to this setup, but is applicable to holographic setups in general. A detailed description of our optical setup has been introduced by the authors of [38].

Three separate grating-stabilized diode lasers (632.9443 nm, 636.2661 nm, and 635.8239 nm) are used. The typical linewidth of the individual laser is specified to be between 0.5 and 1 MHz. With two wavelengths, ${\lambda }_j$ and ${\lambda }_k$, the artificial wavelengths ${\lambda }_a$, as follows,

$${\lambda }_a=\frac{{\lambda }_j·{\lambda }_k}{\left|{\lambda }_j-{\lambda }_k\right|}=\frac{{\lambda }_j·{\lambda }_k}{\left|\Delta \lambda \right|}$$

(1)

can be calculated to 915 µm, 140 µm, and 121 µm. The different lasers are connected through a fiber switch to the interferometer. Only one wavelength is present in the sensor at a time. To ensure a common source point, single-mode fibers are used to connect the lasers (1) to the sensor. The beam is split into reference and object beam by a polarizing beam splitter cube (3). By rotating the angle of the polarizing-maintaining fiber, the ratio between the beams can be adjusted. Later, the orthogonally polarized beams are combined without losses by the second polarizing beam splitter cube. For the reference beam path, an achromatic lens (2) converts the beam to a convergent beam with an interim focus on the piezo (4). Having passed a polarizing beam splitter cube (7), the beam is deflected onto the camera (8). For temporal phase shifting, the piezo introduces a minimum of three phase steps $n$ by an angle of approximately $2\pi /n$, respectively. The piezo runs in an open-loop mode and the exact phase-steps are calculated subsequently (Section 3) [1,39]. Passing a polarizing beam splitter cube (3), the object beam is reflected by the sample (5), and focused onto the camera by an imaging lens (6) as an interference pattern.

Figure 1b shows the measurement data acquisition time series for our three laser setup at three phase steps $n$. With this configuration, the data acquisition for one measurement takes 60 ms and includes nine raw images.

3. Algorithm

The numerical reconstruction of the all-in-focus height map is performed in four steps, as follows:

• Temporal phase-shifting–determination of the global phase shifts between single images induced by the piezoactuator and the calculation of the complex wave [39,40,41].
• Propagation of the complex wave $C_{\lambda }\left(x,y\right)$ and calculation of the synthetic wavefronts [39].
• Shape-from-focus (described in this section):
  • Maximum filter (Section 3.1) [38]
  • Gaussian fit (Section 3.2)
• Iterative combination of focus criterion and finer artificial wavelengths [15,38].

We propose a method to calculate an extended focus image based on the phase noise, and combine the respective synthetic wavefronts using this output of the algorithm.

As input data, the phase information at arbitrary (even multiple) wavelengths can be processed. A pre-filter with radius 2 has been found to effectively reduce camera noise on the complex holograms. Firstly, the phase noise ${\Delta }_x\phi$ and ${\Delta }_y\phi$ in $x$ and $y$ direction of a synthetic wavefront at reconstruction distance $z$, is calculated.

$$\begin{gathered} {\Delta }_x\phi =\left({\mathsf{\phi }}_{x+1}-{\phi }_x\right) mod 2\mathsf{\pi } \\ {\Delta }_y\phi =\left({\mathsf{\phi }}_{y+1}-{\phi }_y\right) mod 2\mathsf{\pi } \end{gathered}$$

(2)

The gradient vector

$$\nabla \phi =\left(\begin{array}{c}{\Delta }_x\phi \\ {\Delta }_y\phi \end{array}\right)$$

(3)

can be normalized as follows:

$$\hat{\nabla }\phi =\frac{\nabla \phi }{\left|\nabla \phi \right|}=\frac{\left(\begin{array}{c}{\Delta }_x\phi \\ {\Delta }_y\phi \end{array}\right)}{\sqrt{{({\Delta }_x\phi )}^2+{({\Delta }_y\phi )}^2}}$$

(4)

With a moving average filter (e.g., radius $r$ = 8), the amplitude of the phase smoothness $s_z\left(x,y\right)$ for each pixel at reconstruction distance $z$ is used as a focus criterion, as follows:

$$s_z\left(x,y\right)=\left|{\displaystyle \sum }_{\begin{array}{c}i=-r\dots r\\ k=-r\dots r\end{array}}\hat{\nabla }\phi \left(x+i,y+k\right)\frac{1}{{\left(2r+1\right)}^2}\right|$$

(5)

Figure 2 illustrates the principle of the algorithm graphically, as follows: Panel (a) shows a synthetic phase map with linearly increasing noise from left to right. Panel (b) shows the anisotropic normalized phase gradient $\hat{\nabla }\phi$ for the low-noise (green) and high-noise (orange) areas, taken from the synthetic data in panel (a), respectively. Areas of low noise are characterized by a uniform phase gradient direction, whereas noisy areas appear as random phase vectors. In the following, we use a filtered sum of these gradient vectors $\sum \hat{\nabla }\phi$, shown in panel (c), as our phase noise or rather, the phase smoothness criteria. It can be interpreted as the filtered information inside the moving average kernel. A large gradient vector sum indicates an area of smooth phase and thus focused pixel. In out-of-focus areas, the gradient vectors are noisy and the $\sum \hat{\nabla }\phi$ is shorter.

Figure 3 shows the output $s_z$ of the algorithm at a single point $\left(x,y\right)$, and different reconstruction distances $z$ for a gear sample. The graph clearly shows a maximum at around $z$ = 6 mm.

To find the in-focus distances within the stack of images at propagation distances $z$, a minimum phase–noise value of the stack of calculated propagation distances $z$ for each pixel coordinate $\left(x,y\right)$ has to be found. In the following, two methods for the synthesis of a phase map with extended depth of field are proposed.

3.1. Maximum Filter

Previously, we used a similar phase–noise algorithm with pixel wise maximum filter data processing [38]. The maximum filter output $z_{opt,max}$ can be calculated with a low computation effort and shows a very even yet discrete focusing map, as follows:

$$z_{opt,max}=\arg ma{x}_z\left(s\left(x,y\right)\right)$$

(6)

3.2. Gaussian Fit

Further investigations qualify the pixel wise Gaussian fit as a tremendous improvement. The results presented in Section 4 could be achieved by the following algorithm: all pixels $\left(x,y\right)$ of the shape-from-focus stack are used as sampling points for a pixel wise Gaussian fit with four degrees of freedom, as follows:

$$f\left(x|a,\mu ,{\sigma }^2,c\right)=a·\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{\left(-\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}\right)}+c$$

(7)

The fit is initialized using the distance of the maximum phase smoothness as a first estimate for the peak position $\mu$, and c being the ground noise of the signal. The scaling parameter $a$ is initialized as the peak value of the maximum phase smoothness, along with the experimentally determined ${\sigma }^2$ of 0.5, complete our initial parameters for up to 1000 iterations. The expected value ${\mu }_{opt}$ of each Gaussian fit is used for the final focus map $z_{opt,Gauss}$.

$$z_{opt,Gauss}={\mu }_{opt}$$

(8)

The values with a maximum phase smoothness below 0.3 are marked in a separate mask. The corresponding height $h$

$$h=-z_{opt}$$

(9)

is used as a rough synthetic wavelength and serves as the starting information for the combination of further synthetic wavelengths.

In order to being able to use this method for inline-measurements, computation-intensive calculations are processed on the GPU, since pixel wise operations can be efficiently evaluated in parallel. Having transferred the complex holograms to the GPU, the phase noise, filtering, and Gaussian fitting are computed on the GPU.

4. Results

Unless otherwise stated, all calculations in this section are conducted on the finest synthetic wavelength at 121 µm.

4.1. Gear Tooth Sample

An evaluation of a gear tooth with an axial extension greater 5 mm is presented in this Section. Thirty equally spaced propagation distances from 0 mm to 10 mm have been calculated. Figure 4 shows our experimental setup, including the gear test sample.

Figure 5 shows the phase $\phi$ (top) and phase smoothness $s_z$ (bottom) maps for two propagation distances $z$ = 3.0 mm and $z$ = 6.0 mm, respectively. For areas of low noise and thus focused parts, the phase shows sharp stripes, whereas the unfocused areas are superimposed by noise. In the left panels, a propagation distance of $z$ = 3.0 mm is pictured with the focused point and the associated phase smoothness on the right area of the measurement. The right panels were evaluated at a propagation distance of $z$ = 6.0 mm and show a clear maximum band and horizontally decreasing focus on the left of the measurement.

As a next step, the synthesis of phase maps with an extended depth of field has to be addressed—the calculated propagation stack holds a best phase–noise distance for each pixel coordinate $\left(x,y\right)$. Figure 6 shows the progression of the phase–noise signal at different reconstruction distances $z$ for equally distributed points on the gear flank. Two thumbnails at $z$ = 3.50 mm (left) and $z$ = 6.33 mm (right) illustrate the position of the crosses on the sample near the focus or the orange cross (left) and the red cross (right).

In the following sections, we propose two methods for the focus point selection with different applications.

4.1.1. Maximum Filter

Figure 7a shows the reconstruction distance at the maximum phase smoothness for each pixel. Since the shape-from-focus algorithm outputs the focus distance to the sample, its height information is inverse to the sample’s real surface. We note the inverse height information in the shape-from-focus output compared with the sample’s real surface. Except for the gear flank edge, the resulting height map shows a very smooth yet discrete focusing map. At the joint of both areas, a focus distance peak can be observed in Figure 7a. This outlier is due to a drop in the amplitude of $s_z$ in areas of low gradients, as shown at the yellow point in Figure 7b, which can lead to the wrong focus plane estimation.

4.1.2. Gaussian Fit

Using the Gaussian fit approach introduced in Section 3, we can overcome the discrete distribution and get the result shown in Figure 8a. The output now shows an even smoother progression with similar outliers at the gear flank edge. Using the amplitude of the phase gradient direction vectors, shown in Figure 8b, we get information about the signal quality, which can be used for weighted filtering.

Gaussian fits based on the sampling points, as described in Section 3, are plotted in Figure 6. The dashed-line shows the initial Gaussian fit guess for each point plotted in Figure 8a. The final Gaussian fits are shown as solid lines and are in very good agreement with the reconstruction distances.

Applied to all pixel values, both focus plane selections result in a continuously focused measurement, as shown in Figure 9.

Now instead of using a very large synthetic wavelength, the shape-from-focus output can be used as rough estimation of the object’s surface shape for the combination of wavelengths. By combining the shape-from-focus data with all three synthetic wavelengths, we get the focused unambiguity-extended 3D results in Figure 10. However, due to the discrete progression of the shape-from-focus estimation, we get various combination errors applying finer wavelengths to the maximum filtered input data as shown in Figure 10a. Maximum filtered height estimation data works perfectly for all-in-focus calculations as shown in Figure 10 but has to be generated with very small propagation step width $z$ in order to suffice as a rough synthetic starting point for wavelength combination. Since this is very time-consuming, the Gaussian fit algorithm should be used instead here.

Applying all the finer wavelengths on the inverse of the Gaussian filter output, we get the focused unambiguity-extended 3D result in Figure 10b. The combined data considerably exceeds the geometrical ambiguity and focus limits, and is almost free of combination errors. $s_z$ is used in both of the height maps to mask out data points, but the shape-from-focus algorithm could not reconstruct safely. Masked data points are shown as white points.

4.2. Steps Sample

In this section, a tilted sample with discontinuous height steps larger than the measurement, the unambiguous range is evaluated. Figure 11a shows a photograph of this part, including a mm scale, and Figure 11b shows the sensor setup with the tilted step sample and its step height of 1.2 mm.

Figure 12a shows the shape-from-focus output using the superior Gaussian fit approach from Section 3.2. We were able to fully reconstruct the tilted sample very smoothly. However, because of two reasons, the edges of the plateaus show artefacts, as follows: Within the filter radius, we get mixed phase information and thus mixed $s_z$ values. Additionally, at very steep slopes, strong artefacts can be observed due to the stripes not being sufficiently sampled. These effects can be suppressed by additional $s_z—$weighted filtering.

Figure 13 shows the combined height information for both filtering approaches. Even in a complex sample crossed by many steps far beyond the measurement’s unambiguous range, we get focused unambiguity-extended height information. For the maximum filter approach, the height map, as shown in panel (a), is disturbed by many combination errors. Conversely, the Gaussian filtered height map in panel (b) is almost free of combination errors.

5. Discussion

We presented a shape-from-focus algorithm for multi-wavelength holography using a phase noise measure, based on phase gradients. Firstly, a generic measuring setup for the application of this method was introduced. Two approaches (pixel wise maximum filter and the Gaussian fit on the shape-from-focus stack output) for the all-in-focus measurement data calculation were presented subsequently. The output of the algorithm is further used as a rough height estimation to overcome the phase ambiguities, and allows the extension of the unambiguous measurement range.

We were able to calculate an all-in-focus measurement and extend the range of unambiguousness beyond 12 mm at the largest synthetic wavelength of only 915 µm. In this range, we could not observe a decrease in signal quality. To date, no fundamental limit to the possible range are known. Using a setup with collimated illumination, experiments at meter-scale distances would be possible.

After having explained the algorithm in general, their respective performance at complex free-form surfaces was demonstrated. A gear tooth flank and a step sample at steep viewing angles demonstrate the performance of the algorithm. Both filters are able to reconstruct an all-in-focus phase map. However, only the Gaussian filter approach is able to reconstruct an unambiguity-free height map on top of the synthetic measurement data without combination errors.