Only the basic steps of the slanted-edge method are introduced in the ISO 12233 publication. The analysis in the previous section shows that Bayer filter color cameras have higher requirements in terms of the accuracy of the algorithm when performing MTF measurements. Therefore, this paper optimizes and improves the three key steps of the detection of the slanted edge, the acquisition and processing of the ESF, and the acquisition and processing of the LSF.
3.1. Detection of Slanted-Edge
The detection of the slanted edge is an important part of the slanted edge method. The user must be able to accurately detect the position of the slanted edge of the image to ensure the accuracy of the subsequently acquired ESF and LSF and the calculation of
MTF. The ISO12223 method uses the centroid method to calculate the position of the centroid of each row of pixels and then uses the least squares method to fit a straight line. The equation for the centroid method can be expressed as
In this equation,
i and
j represent the row and column of the image, respectively.
Centroid(
j) represents the position of the centroid of a certain row of pixels, and
g represents the gray value of a certain pixel. When the image is not affected by noise, the ISO 12233 method yields a smaller error in the detection of the slanted edge. However, when there is noise in the image, the error of detection is relatively large due to the reduction in the calculation accuracy of the position of the centroid. The image simulated via the ISO 12233 method for the detection of the slanted edge is shown in
Figure 5. The size of the image is 100 × 140 pixels. When generating a tilted edge, it is necessary to rotate the image, but it is difficult to accurately control and calculate the angle of the rotated edge. Therefore, we choose a slope of 1 to generate the simulation image. The digital number (DN) value of the bright edge is 200, and the DN value of the dark edge is 50. The degraded image is obtained using a Gaussian kernel function with a variance of 0.5 as a low-pass filter. Noise is added in
Figure 5b, and the signal-to-noise ratio (SNR) of the image is 20 dB. When using the ISO 12233 method to detect the slanted edge of the image before and after adding noise, the fitting results of the slope are 1.0009 and 1.0232, respectively. It can be seen from the simulation results that the ISO 12233 method yields a small error when detecting the slanted edge of a noise-free image, and the result is poor when detecting the slanted edge of an image with noise.
The Hough transform is less sensitive to noise, and it is one of the main methods used for the detection of straight lines. The basic principle of the Hough transform is to transfer the detection of straight lines in image space to the detection of points in parameter space. The dual transformation of image space and parameter space can be expressed as
where
ρ represents the vertical distance of the line to the origin, and
α represents the angle between the
x-axis and the vertical line of the line.
As shown in
Figure 6, this paper uses the Hough transform to transfer the detection of a straight line in the rectangular coordinate system to the detection of a point in the polar coordinate system. The points on a straight line in the rectangular coordinate system correspond to curves intersecting at a point in the polar coordinate system, and the line corresponding to the slanted edge is the point with the highest number of intersections in the polar coordinate space. The Hough transform needs to traverse all pixels, and
α needs to be calculated using Equation (3) within the range of [−90°, 90°]. The smaller the step size of
α, the higher the accuracy of line detection. Therefore, the Hough transform involves a high burden of calculation and takes a long time. In order to solve this problem, in this paper, we use the Canny operator to preprocess the image, preliminarily determine the location of the slanted edge, and improve the speed and accuracy of the Hough transform for line detection.
The preprocessing of the image by the Canny operator proceeds via four main steps: the smoothing of the image, the calculation of the gradient magnitude and direction, the suppression of gradient magnitude, and the removal of false edges. The Canny operator uses Gaussian filtering to smooth the image, and the variance is the scale parameter of the Gaussian filtering, which determines the smoothness of the Gaussian filtering window. The selection of the scale parameter is very important. The smaller the scale parameter, the higher the positioning accuracy, but the worse the noise processing effect. With a larger scale parameter, although the noise in the image can be better processed, the information of the slanted edge will become a high-frequency signal and will be easy to lose. When the Canny operator uses the non-maximum value to suppress the gradient magnitude, the selection of the threshold parameter is very important. If the threshold parameter is too high, the information on the slanted edge will be lost, and if it is too low, more false edges will appear. Therefore, in order to improve the accuracy of the Canny operator for image preprocessing, this paper proposes to use the adaptive function and Otsu method to achieve the optimal selection of scale and threshold parameters.
When the pixel point is a noise point, the scale parameter should take a larger value for smoothing; when the pixel point is a smooth area point, the scale parameter should take a smaller value to make the gray value change less; when the pixel point is a slanted-edge, it should lie between the above two cases. Therefore, the adaptive function is constructed in this paper as follows:
where
σbest represents the optimal scale parameter corresponding to the current pixel point,
g(
i, j) is the gray value of a current pixel point, and
m and
n represent the sizes of the filtering window. The basic principle of the Otsu method is to divide the image into two categories according to the gray features of the image and to determine the optimal threshold for suppressing the gradient amplitude in the Canny operator by finding the maximum interclass variance. Suppose the grayscale range of the image is {1, 2, ….
v}, and suppose the optimal threshold is
H. We divide the pixels into two classes,
C1 = {1, 2, ….
H} and
C2 = {
H + 1,
H + 2,
…. v}, and the interclass variance of
C1 and
C2 can be expressed as
In the equation, P1 = N1/N0 and P2 = N2/N0. N0 represents the total number of image pixels. N1 and N2 represent the number of pixels in the two types of images, respectively. μ0 represents the mean value of the gray level of the entire image. μ1 and μ2 represent the mean value of the gray levels of the two types of images. When the interclass variance IV takes the maximum value, the optimal threshold H is obtained. We take this threshold as the high threshold of the Canny operator, and we take half of the high threshold as the low threshold.
The results derived by the original Canny operator, and the Canny operator based on adaptive filtering and the Otsu method, when processing the noisy images shown in
Figure 5 are shown in
Figure 7. From
Figure 7, we can see that the edge extraction effect of the original Canny operator is poor. The improved Canny operator in this paper has a very good processing effect, and it can remove false edges while retaining slanted-edge information.
In order to verify the accuracy and stability of the improved algorithm in the detection of the slanted edge, this paper simulates and analyzes the detection accuracy of the slanted edge under different SNRs via four methods: the ISO 12233 method, the method combining adaptive filtering and the ISO 12233 method, the Fermi function method (which is currently being used more often), and the method combining the improved Canny operator and the Hough transform. The parameters of the simulated image are the same as those in
Figure 5. In order to reduce the random error caused by the added noise of the computer, an image with the same SNR will be detected 1000 times by each method. The relative errors of each method are shown in
Table 1.
From the results in
Table 1, we can see that the accuracy of detecting the slanted edge using each of the four methods is very high when no noise is added. When noise with different SNRs is added, the accuracy of all four methods is reduced. Among them, the accuracy of the ISO 12233 method is the most affected by noise. After adaptive Gaussian filtering, the ISO 12233 method is less affected by noise. The accuracy of the Fermi function method is higher than that of the ISO 12233 method after adaptive filtering. The improved method in this paper yields the highest accuracy and is the least affected by noise. Therefore, the improved slanted-edge detection algorithm can provide a solid guarantee of the high precision required by Bayer filter color cameras.
3.2. Acquisition and Processing of ESF
The ESF can be obtained by processing the image according to the results obtained from the detection of the slanted edge, which is based on the light intensity distribution of the image with the slanted edge. The approach to the acquisition of the ESF can be divided into parametric and non-parametric methods, according to whether a mathematical model is used. The parametric method involves using the function model to directly fit the image data, and this places higher requirements on the model. However, it is not easy to obtain a high-precision function model in practice, and due to the influence of noise factors, the parametric method often has a poor fitting effect or may even encounter non-convergence in the application process. The non-parametric method directly processes the data of each line of the image and has greater adaptability. In this paper, we chose the projection method for ESF acquisition, which is also used in the ISO 12233 method. As shown in
Figure 8, the basic principle of the projection method is to use the characteristic of a small phase shift between different scan lines formed by the angle between the slanted edge and the scan direction and project the data in each line in the direction of the slanted-edge; the ESF is then obtained by averaging the data within the same phase shift period.
The following relationship can be obtained from
Figure 8:
where
m represents the sampling rate of the ESF and
θ represents the angle of the slanted edge. From the above equation, we can see that when the sampling rate of ESF is not an integer, the data in the ESF are uneven, which will affect the accuracy of LSF extraction and
MTF calculation. In the ISO 12233 method, it is suggested that the sampling rate of the ESF should be an integer, but due to the accuracy of the algorithm and hardware, it is difficult to achieve uniform sampling of the ESF. In addition, the noise will prevent the ESF curve from being smooth, which will reduce the calculation accuracy of the
MTF. Currently, methods used for smoothing, such as mean filter, median filter, and Gaussian filter, cannot preserve the motion trend of the signal well. Therefore, in order to solve the above problems and realize the uniformization and smoothing of the ESF data, this paper proposes a method combining cubic spline interpolation and SG filtering to process the ESF.
The cubic spline Interpolation algorithm divides two adjacent data points into
n intervals, and the interpolation function on each interval is a cubic equation. After the ESF curve with continuous curvature is obtained, evenly distributed ESF data can be obtained by uniform sampling on the smooth curve. The calculation method is as follows:
In the above equation, Si(x) = aix3 + bix2 + cix + di. The undetermined coefficients ai, bi, ci, and di on each small interval can be determined according to the continuity condition and the smooth curve condition.
The SG filter divides the ESF data into countless local small areas and performs polynomial least squares fitting on each local area to ensure that the shape of the data curve remains unchanged while filtering out noise. Assuming that the data of a certain local area in the ESF
x(
n) are
x(
i),
i = −
m, …, 0, …,
m, the
h-order polynomial is used to fit these local data.
Using the least squares method to solve the coefficients of the above polynomial, the sum of the squares of the deviations between the fitted curve and the data points is
When the derivative of Er is 0, Er takes the minimum value, and the expression of the function of fi can be obtained by substitution. The value of the fitted polynomial in the center point coordinates is the resulting value of the SG filter at that center point, and the smoothed result of all ESF data can be obtained by continuously moving the frame of the SG filter. In order to retain the trend of the original ESF curve as much as possible, this paper will use a fifth-order polynomial for fitting, with a frame length of 11.
In order to verify the effectiveness of the improved ESF processing method using cubic spline interpolation and SG filtering, a simulation analysis is conducted in this paper, and the parameters of the simulated images are shown in
Table 2. Firstly, we use the improved method described in the previous section to detect the slanted edge, and then use the ISO 12233 method and the improved ESF processing method in this paper to process the ESF data; finally, the accuracy of the different methods can be evaluated by the root mean squared error (RMSE) of the calculated and theoretical values of the
MTF. The simulation results yielded when the SNR is 20 dB are shown in
Figure 9, and the RMSEs of
MTFs of different methods with different SNRs are shown in
Table 3.
It can be seen from
Figure 9 that the original distribution of ESF data is non-uniform, and the data points can be uniformly distributed using cubic spline interpolation. Using SG filtering can smooth the ESF curve under the premise of ensuring the trend of the curve. As shown in
Table 3, when the SNR is 15 dB, the ISO 12233 method fails to complete the calculation of the
MTF curve due to the serious influence of noise, even though the detection algorithm with the improved slanted edge described in the previous section is used. After ESF is processed by the combination of cubic spline interpolation and SG filtering, as proposed in this paper, the calculation of the
MTF curve is successfully completed when the SNR is 15 dB. Moreover, the method proposed in this paper achieves greater accuracy when the SNR is different. Therefore, the processing method of ESF proposed in this paper not only broadens the scope of application of the slanted-edge method but also improves the measurement accuracy of
MTF.
3.3. Acquisition and Processing of LSF
Discrete data with uniform distribution will be obtained after processing the ESF in the previous section, and then the LSF can be obtained from the difference operation. Although the SG filtering of the ESF in the previous section reduces the noise to a certain extent, the differential operation will expand the influence of the noise, so the influence of the residual noise on the
MTF results cannot be ignored. Therefore, the LSF needs to be smoothed or fitted to reduce the effect of noise. The LSF is smoothed using the Hamming window in the ISO 12233 method; the expression of the Hamming window is
where
N + 1 represents the length of the window. The Hamming window can effectively reduce the oscillation noise caused by the differential operation, but when the noise is large, the improvement in accuracy yielded using the Hamming window for smoothing is limited. Since PSF is the intensity distribution function of single-point imaging, LSF involves the sampling and superposition of PSF in a certain direction, and the distribution of PSF can be expressed by the Gaussian function; this paper proposes a Gaussian function fitting to process LSF.
Here, A represents the peak value, μ represents the mean value, and σ represents the variance.
Hamming window smoothing and Gaussian function fitting are separately performed on the results shown in
Figure 9c, and then the
MTF is obtained by Fourier transform. The simulation results are shown in
Figure 10.
It can be seen from
Figure 10a that the smoothing of the Hamming window can reduce the influence of noise to a certain extent, but when the noise is large, the processing effect of the Hamming window is limited. Using the Gaussian function fitting method proposed in this paper to process LSF, the obtained accuracy of the
MTF is relatively high.