1. Introduction
The detection of surface defects is one of the main items of surface quality evaluation for optical elements. In optical systems, scattering induced by surface defects will reduce the utilization efficiency of energy or even lead to severe damage to those with high-energy lasers, such as the inertial confinement fusion system (ICF) [
1,
2,
3]. With the wide application of the optical elements and the development of manufacturing technology, the requirements for defect detection become higher. On the one hand, sufficient detection sensitivity for small-size defects is required to ensure the effectiveness and safety of optical elements for use. On the other hand, to obtain a comprehensive evaluation report containing hazard assessment and cause analysis of defects, the ability of quantitative detection, including defect size metrology and defect classification, has become increasingly significant.
Various detection approaches for surface defects of optical elements have been studied. Commercial microscopes, such as atomic force microscope (AFM) and scanning electron microscope (SEM), are used to detect and characterize the surface defects on fine optics [
4,
5]. These methods have high detection accuracy and sensitivity but require long inspection times. Because of the advantages of non-destructiveness and high throughput, optical inspection techniques have gained more and more attention. For example, Choi et al. [
6] proposed a defect inspection method which obtains defect images using the photo-thermal reflectance technique. Rainer et al. [
7] adopt the total internal reflection (TIR) technique with back and edge illuminations to detect both the surface defects and sub-surface damages of large optical elements. At present, the most common detection approaches for surface defects are based on the dark-field scattering technique [
8,
9,
10]. On this basis, a surface defect evaluation system (SDES) has been proposed and established from our previous works [
11,
12,
13]. It employs subaperture scanning, image stitching, and image processing methods to finally realize the quantitative inspection of surface defects down to micrometer-level dimension. The measuring aperture can reach the order of a meter. However, because the SDES uses a zoom microscope with a long working distance, the numerical aperture of the objective is relatively low. As a result, only a small amount of scattered light induced by defects can be collected, and therefore, especially for small-size defects (such as shallow scratches), the defect signal acquired by the image sensor will be weak [
14]. Hence, the detection sensitivity is limited. Moreover, the SDES lacks the ability to distinguish the defects that exhibit similar features in the acquired image [
15].
In detection approaches based on dark-field scattering techniques, enhancing the defect scattering signal is an effective way to improve the detection sensitivity [
16]. For this purpose, a spot scanning surface defect evaluation system (SS-SDES) is established in this paper. A short-wavelength laser beam is focused on the optical surface to form an illuminated spot, and a positioning system is used to bring the spot onto each portion of the surface along a spiral trace. In each illuminated spot, the scattered light caused by defects will be collected by a large solid-angle ellipsoidal mirror, and then detected by a high-sensitivity photomultiplier (PMT), thus improving the intensity of the acquired defect scattering signal. Additionally, the SS-SDES incorporates a polarization measurement channel, which is capable of measuring the polarization characterization of defects for classification. Due to the use of the spot scanning technique, the SS-SDES cannot, like an image sensor-based system, acquire a defect image directly. Therefore, before defect extraction and analysis, the SS-SDES needs to perform image reconstruction on the acquired intensity signals. However, the existing system deviations, such as the position deviation of the illuminated spot, will cause distortions and even a missing area in the reconstructed image, which will affect the detection results in many aspects and hence degrade the reliability of the system. Although there have been several studies on distortion correction [
17,
18], they are mostly aimed at the optical distortion caused by imaging lenses and cameras, so they are not applicable to the situation in this paper.
To address these issues, this paper develops a system calibration method based on straightness and scale constraints and an image reconstruction method based on weighted average. First, the system deviations are parameterized to model the practical scanning trace, which will be used to compute the image coordinate of each acquired intensity signal. On this basis, a calibration plate with specific defect patterns is employed as the test sample, and several constraints are established to constrain the image coordinates corresponding to these defects. Then, an optimization is implemented to minimize the constraints and hence to obtain the optimal estimate of the system deviations. This estimate is subsequently used to adjust the system and establish an accurate mapping relationship for image reconstruction. The adjusted system can perform the scan for the whole optical surface in a normal way but without any missing area. Finally, using the proposed image reconstruction method, a complete defect image with low noise and low distortion can be obtained; this will greatly improve the detection accuracy and reliability of the system. The proposed methods are applicable to any other detection systems based on spot scanning, such as those for wafers, films, and rough surfaces.
The remainder of this paper is as follows.
Section 2 introduces the configuration of the SS-SDES and the ideal image reconstruction and then summarizes the existing system deviations as well as their impacts. In
Section 3, the modeling of practical scanning trace and the details of the proposed system calibration method and image reconstruction method are described.
Section 4 shows the experiment and comparison results, and
Section 5 presents a conclusion of this paper.
3. System Calibration and Image Reconstruction Methods
To address the problems introduced by system deviations, this paper proposes a novel system calibration method based on straightness and scale constraints and an image reconstruction method based on weighted average. The methods will be described in detail below, including the following: (1) how to parameterize the system deviations and model the practical scanning trace; (2) how to calibrate and obtain the system parameters; and (3) how to reconstruct a quality image.
3.1. Modeling of the Practical Scanning Trace
Figure 4 presents the schematic of practical system state at initial time (
) and scanning time
. With the help of this figure, this section will parameterize the system deviations described above and finally derive the coordinate of the illuminated spot
to establish a mathematical model of the practical scanning trace of the SS-SDES.
According to the definition of the system coordinate system, the rotation axis
is a straight line that passes through the origin
.
can be described, using two parameters, by the symmetric equation of the line in space:
where
and
are the polar and azimuthal angles of the line as shown in
Figure 4a.
The optical surface is a plane that passes through the origin
. Let
be the plane equation of the surface at scanning time
. At the initial time (
), the surface is horizontal, meaning that the normal vector of
is
. At any scanning time
, the plane equation
can be obtained by rotating
counterclockwise about
by an angle of
as
where
is the
rotation matrix corresponding to a rotation by an angle of
about the fixed axis
, which can be computed by Rodrigues’ rotation formula [
21].
Since the illuminated spot is the intersection of the illumination light and the optical surface, it is required to parameterize the illumination light by a line. Let
be the equation of the illumination line at scanning time
:
Four parameters are included in the equation, where
and
are the polar and azimuthal angles of the illumination line, and
is the coordinate of the initial illuminated spot. By solving the combined equation of Equations (4) and (5), one can obtain the coordinate of
in the system coordinate system, denoted as
; this is subsequently transformed into the coordinate in the surface coordinate system by
where
is a
matrix consisting of the first two rows of the rotation matrix
.
By substituting the surface coordinate of the illuminated spot,
, into Equation (2), one can finally obtain the mapping relationship from the sequence number
to the image coordinate
. For brevity, we use
to denote the mapping relationship, and therefore, the mapping function can be written as
where
is the union of all parameters involved in the modeling, called the system parameter vector, which determines the mapping relationship
. In the practical implementation, the mapping function, Equation (7), is calculated by the following 7 steps:
- Step 1:
Calculate the corresponding scanning time of the sequence number, as ;
- Step 2:
Define the line equation of the rotation axis using Equation (3) with parameters and ;
- Step 3:
Calculate the plane equation of the sample surface after rotation about the rotation axis using Equation (4) with the parameter ;
- Step 4:
Define the line equation of the illumination line using Equation (5) with parameters and ;
- Step 5:
Calculate the intersection point of the sample plane and the illumination line;
- Step 6:
Transform the coordinate of the intersection point from the system coordinate system to the surface coordinate system using Equation (6) with parameters , and ;
- Step 7:
Transform the surface coordinate of the intersection point to the image coordinate using Equation (2) with parameters and .
It can be found that the ideal scanning trace, introduced in
Section 2.2, is a special case of the practical scanning trace with the ideal system parameter vector
, where all the parameters are the setting values of the system, and the specific expression of the ideal mapping relationship
is given by Equations (1) and (2). Apparently, to reconstruct an undistorted image, an accurate system parameter vector is required for establishing the actual scanning trace and mapping relationship.
3.2. System Calibration Method
A system calibration method capable of getting the accurate system parameter vector is studied here. In this method, a specific calibration plate with a group of line-like scratches and distance-marked dot pairs on its surface is employed as the test sample. The distance-marked dot pair refers to a pair of point-like defects, such as digs, between which the distance is known. As can be found in
Figure 5, this method can be divided into three major steps. The first step is to extract the feature sequence number sets, which consist of the sequence numbers of intensity signals that come from the defects. In the second step, a constraint function for constraining the image coordinates mapped from the feature sequence number sets is defined, and the third step is to perform a nonlinear optimization to minimize the constraint function, so that the optimal system parameter vector can be obtained. More details about these three steps will be given in the following.
3.2.1. Extraction of Feature Sequence Number Sets
The discrete intensity sequence acquired by the SS-SDES is one-dimensional. For each intensity signal in the sequence, although its magnitude can be used to roughly distinguish whether it is a defect signal, directly determining the location and characteristics of the defect to which the signal belongs is not possible. Hence, for the extraction of the sequence numbers corresponding to specific defects on the calibration plate, an image is first reconstructed using the ideal mapping relationship
, called the original image. Then, image processing techniques are performed to find out the specific scratches and distance-marked dot pairs and to subsequently extract their feature points, which indicate the centroid for digs and the points on the skeleton for scratches. Finally, the sequence numbers corresponding to these feature points are obtained through a backtracking strategy: the discrete intensity sequence is traversed again to calculate the image coordinate for each sequence number; if the distance between the coordinate and any a feature point is less than 0.5 pixel, the sequence number is considered to correspond to this feature point. In addition, multiple groups of scratches and distance-marked dot pairs are selected to make the feature points locate at every part of the image as much as possible. Let
be the number of selected scratches and
be the number of selected distance-marked dot pairs. The obtained feature sequence number sets associated with them can be expressed by Equations (8) and (9), respectively:
where
is the sequence number, and
represents the number of sequence numbers in the
th scratch, and
represents the actual distance between the
th distance-marked dot pairs.
3.2.2. Establishment of Constraint Function
For an arbitrary system parameter vector
, the mapping relationship
can be obtained according to the practical scanning trace model established in
Section 3.1. By using
, one can compute the image coordinates of each feature sequence number set and then observe the appearance of the corresponding defect in the reconstructed image. To avoid image distortions, our method will place two types of constraints on these coordinates. First, straight scratches should not be bent in the reconstructed image, meaning that all sequence numbers in each
are expected to map to image coordinates which are collinear. Hence a straightness constraint is required. As is known, the distance of a point
to a line determined by two endpoints
and
is
, where
is the unit normal vector of the line.
and
denote the operations that return the absolute value of a scalar and the L2-norm of a vector, respectively. For the
th selected scratch, assuming that its two endpoints correspond to the first and last sequence numbers in
, the straightness constraint for
can be defined by the maximum straightness error
as
Second, scaling is not permitted in the reconstructed image, otherwise the relationship between the pixel size and the actual size will be changed, making the extracted geometric features of defects inaccurate. As described in
Section 2.2,
, the actual size of each pixel in the reconstructed image, is a fixed preset value. To maintain this characteristic, a scale constraint, for constraining the two sequence numbers in each
to map to two image coordinates between which the distance is equivalent to the actual value
, is constructed by the distance error
, whose expression is given by
Finally, the total constraint function is defined as a weighted sum of the squares of straightness constraints and scale constraints:
where
is the weight factor used to adjust the relative importance between the two types of constraints.
3.2.3. Nonlinear Optimization
Since
characterizes the distortion of the reconstructed image, the optimal estimate of the system parameter vector, called the calibrated system parameter vector
, can be obtained by minimizing it:
This is a nonlinear least-squares problem. As the number of variables increases, the problem becomes more complex. In the modeling of the practical scanning trace, it can be found that for the three variables,
,
, and
, the mapping relationship is actually related only to
and
. Therefore, the sampling frequency
can be fixed to the setting value with only
and
to be optimized, thus reducing one variable. To address this issue, the Levenberg–Marquardt Algorithm [
22] is employed after comparison to other optimization algorithms. It requires an initial guess of
which can be set as the ideal system parameter vector
.
3.3. System Adjustment and Image Reconstruction Method
Once the calibrated system parameter vector
is obtained, the corresponding calibrated mapping relationship
can be established to reconstruct an image free of distortions. However, because of the position deviation between the initial illuminated spot and the origin, the undetected area is still present at the central part of the surface and the image. Thus, it is required to adjust the system to make them coincide. The adjustment method is to move the two-dimensional translation stage from its original position by distances of
and
along the X and the Y directions, respectively, and take this position as the starting point of scanning. The values of
and
are dependent on the thickness of the sample to be detected. Let
be the thickness difference between the sample and the calibration plate. When
, the origin
, which is the intersection of the rotation axis and the surface, will shift, leading to the changes of the coordinate of the initial illuminated spot. Assuming that the sample’ surface is always on the focal plane of the ellipsoidal mirror, we can derive the expression of
and
using simple geometric relations as
where
and
are the parameters that represent the coordinate of the initial illuminated spot in
.
After adjustment, the SS-SDES can perform the scan for the whole optical surface in a normal way but without any missing area at the center. Let be the system parameter vector of the adjusted system. Apparently, is the same as except that the values of parameters and are equal to zero. The adjusted mapping relationship is then constructed to map the discrete intensity sequence acquired by the adjusted system to image coordinates. This is a mapping transformation from one dimension to two dimensions, and there still exist some problems to be addressed. First, the resulting coordinates are generally non-integral, failing to meet the regulation that requires the image coordinate of each pixel to be integral. Second, because the sampling frequency and the scanning velocity are fixed during scanning, the density of sampling points in the central area of the surface will be much higher than that in the edge. As a consequence, the pixel in the central part of the reconstructed image may correspond to more than one sequence number while some pixels in the edge correspond to zero. Aiming at these problems, this paper proposes an image reconstruction method based on weighted average as follows:
Initialize an image matrix and a weight matrix with the same size determined by the actual scanning range and ; initialize the sequence number ;
Calculate the image coordinate corresponding to the sequence number by , and round it to the integer coordinate ; define a weight value according to the distance between them as ;
Update the image matrix by with , the corresponding intensity in the acquired discrete intensity sequence, and the weight matrix by ;
Update the sequence number by ; if the traversal has not been completed (i.e., ), go back to step 2;
Compute the element-wise quotient of the two matrices, i.e., ;
Find the positions at which the elements in have the value of zero and fill the elements at these positions in (no-value pixels) using the average of 8-neighborhood.
The output is the ultimate reconstructed image. This method can not only solve the problems of non-integral coordinates and no-value pixels, but also suppress the signal noise through the weighted average strategy. For any detection system based on spot scanning, this method can effectively improve the quality of the reconstructed image.
5. Conclusions
A spot scanning surface defect evaluation system, which has better performances on the detection for small defects and defect classification than the traditional systems, is established. However, subject to the existing deviations, the system cannot directly provide a high-quality defect image. This paper develops research on these system deviations and then establishes the mathematical model of the practical scanning trace. On this basis, a system calibration method based on straightness and scale constraints is proposed. This method employs a calibration plate with specific defect patterns as the test sample, and places particular straightness constraints and scale constraints on the defects after being extracted. By minimizing the constraints, one can obtain the optimal estimate of the system deviations, which is used, on the one hand, to adjust the system for eliminating the position deviation of the initial illuminated spot, and on the other hand, to establish an accurate mapping relationship for reducing the distortion of the reconstructed image. The adjusted system can perform the detection for the whole optical surface in a normal way but without any missing area at the center. Additionally, an image reconstruction method based on weighted average, capable of eliminating no-value pixels and suppressing signal noise, is proposed, thus enhancing the quality of the reconstructed defect image. Finally, experiments demonstrate the effectiveness of the proposed methods, with the results that the maximum straightness error is within 1.8 pixels and the scale error is within 0.7 pixels. Our methods provide the defect image data with high quality and high precision for the following image processing to extract defects, and hence lay the foundation for improving the detection accuracy of the system. The research results of this paper can also be applied to similar problems of other detection systems based on spot scanning, such as those for wafers, films, and rough surfaces. In the future, more constraints will be considered to improve the accuracy of the system calibration method. Our system can be further improved in two ways: (1) The raw data acquired by the system contains higher-resolution information, which can be utilized to improve the detection accuracy. (2) Collecting and detecting the specular reflection light can provide additional potentials to system adjustment and defect detection.