1. Introduction
With the development of the precision manufacturing industry and the increasing demand for technical indicators, micropore array structures combining the macro and micro scales have emerged in various high-performance devices, such as micropore arrays on aircraft engine injection disks with over 4700 micropores. The shape and size of the pores have a significant impact on the performance of the parts [
1]. In the process of processing and usage, there may also be defects on the surfaces of micropores. The quality of microporous structure processing will determine the quality of the components, so the rapid detection of parameters is of great significance. Image methods, due to their ability to quickly obtain data, have great potential for use in visual measurement or multi-sensor fusion parameter measurement.
For a spatial aperture, when the imaging plane is not parallel to the plane where the aperture is located, the projection of the formed image on the camera imaging plane takes the form of an ellipse. The shape and size of the circular hole can be transformed from an ellipse, which means accurate acquisition of elliptical parameters is crucial. There are three general ellipse fitting algorithms, namely, the method of least squares, the method based on Hough transform, and the method based on arc segment extraction.
Ellipse fitting based on the least squares method is an optimal estimation technique that uses the maximum likelihood method to minimize the sum of squares of measurement errors when the random error is normal distribution. However, it is greatly affected by outliers, so the focus of improvements is to remove outliers. Li [
2] used the double segmentation method to segment two overlapping ellipses to obtain two sets of effective ellipse edge points. Maini [
3] proposed an improved ellipse direct least squares fitting method, which utilized normalization techniques and introduced disturbance and resampling strategies to make it superior in terms of robustness, but it increased the computational burden. Liang [
4] constructed a new non-convex nonlinear optimization formula to improve the ability to suppress outliers, but it also has the problem of low computational efficiency. Azarakhsh [
5] proposed a real-time ellipse detection method, which first extracts all contours from the input image using the ad hoc method, and then performs ellipse fitting on the contours using the least squares method. However, this method achieves lower accuracy in detecting elliptical targets with short distance.
The second ellipse-fitting method is the method based on Hough transform. Its basic idea is consistent with Hough line detection, which first maps the data to a specific parameter space and then votes to obtain the ellipse parameters. However, due to the fact that the ellipse has five free parameters, the time and memory costs are unacceptable. Some scholars have also proposed improved Hough ellipse detection algorithms. The usual approach is to use the geometric properties of ellipses and edge direction information for parameter decomposition [
6,
7,
8], but this still requires a lot of time and memory. Xu [
9] proposed a random Hough transform method to reduce the memory requirement. Some scholars have applied this in ellipse detection [
10,
11], which requires five points to determine the ellipse parameters. Chen [
12] Yanxin proposed a random Hough transform ellipse detection method that only needs three points to determine the ellipse parameters, but it is still time-consuming. Subsequently, Kwon [
13] proposed a method for the rapid detection of ellipses using the geometric characteristics of three points on an ellipse, using the normal and differential equations of the ellipse. This method achieved good results in both detection reliability and computational speed. Yasuaki [
14] decomposed the parameter space in the Hough transform into each parameter, and performed serial calculations on each parameter, which were implemented on a GPU system to reduce computational time and space.
The third ellipse fitting method is based on arc segment extraction, which has been proposed in recent years, and the basic idea is to replace discrete edge points with a set of continuous edge points named arc segment that form an ellipse as ellipse detection data. Prasad and Leung used elliptical arcs to fit ellipses, and through fitting calculations, the center of the ellipse can be obtained [
15]. Afterwards, Wang adopted a top-down strategy by merging edge points into ellipses and using an integral chain to accelerate the fitting process [
16]. Lu proposed a method based on arc-supported line segments, which uses the Line Segment Detector (LSD) algorithm for arc segment extraction and has high detection accuracy [
17,
18]. Meng used the arc segment adjacency matrix to obtain a method for combining all arc segments. This construction method will consume a lot of computer memory [
19].
Due to the existence of defects, the current ellipse detection methods have significantly worse fitting performance and poor robustness. Moreover, in order to ensure the accuracy of visual measurement, the size of the image obtained by vision is too large. This greatly increases the time and cost. For the combination of array structure vision and probe measurement, the imaging time of a single hole feature in the field of view is limited [
20], so there is a requirement related to the fitting time of elliptical features. Moreover, the existing processing methods do not fully consider the influence of elliptical defects and interference points, resulting in a decrease in the accuracy of ellipse fitting. As such, this article proposes an edge repair algorithm based on the approximation principle of circles to repair defect ellipses, thereby proposing outliers, and the image repaired by the least squares method is used for ellipse fitting, which can obtain ellipse parameters quickly and accurately.
2. Measurement Principles and Analysis
2.1. Camera Imaging Model
For the aperture to be measured, the camera is first used to obtain an image of the micropores. The projected image is an ellipse on the camera’s imaging plane. To obtain the geometric information of the micropores, a perspective projection model of the geometric correspondence between the image points and the spatial object points needs to be established. The projection position p of any point P on the micropore in the image is the intersection of the line connecting the optical center O and point P with the image plane, as shown in
Figure 1.
O
w-X
wY
wZ
w is the world coordinate system. O
c-X
cY
cZ
c is the camera coordinate system. O
1-xy is the imaging plane coordinate system. O
o-uv is the pixel coordinate system. The coordinates of point P in the world coordinate system are (X
wp, Y
wp, Z
wp). The coordinates in the camera coordinate system are (X
cp, Y
cp, Z
cp). The coordinates of point p in the imaging plane coordinate system are (x
p, y
p). In the pixel coordinate system, the coordinates are (u
p, v
p). The formula of conversion from the world coordinate system point P
w (X
w, Y
w, Z
w) to the camera coordinate system imaging point P
c (X
c, Y
c, Z
c) is expressed as the following formula, where R is a rotated orthogonal matrix and T is a translation matrix:
In the camera coordinate system, the optical center O
c of the camera is the origin, the Z
c axis is the optical axis, the X
c axis is parallel to the imaging coordinate system’s x-axis, and the Y
c axis is parallel to the imaging coordinate system’s y-axis. The vertical distance from the optical center O
c to the imaging plane O
1−xy is the effective focal length f. According to the projection principle, the conversion model between the camera coordinate system and the imaging coordinate system can be expressed as follows:
The matrix form can be expressed as follows:
The coordinates of the origin O
1 of the imaging plane coordinate system in the pixel coordinate system are (u
0, v
0), where dx and dy, respectively, represent the actual size of each pixel in the x and y directions. The point coordinates in the pixel coordinate system can be represented as follows:
The matrix form of homogeneous coordinates is:
From (1) to (5), it can be concluded that the relationship between the camera coordinate system and the pixel coordinate system is:
Next, we can obtain the correspondence between the world coordinate system point Pw and the pixel coordinate system point p:
We then normalize the focal lengths in the x and y directions in the equation, where fx and fy are equivalent focal lengths (fx = f/dx, fy = f/dy).
2.2. Ellipse Fitting Based on Least Squares Method
Ellipse fitting based on the least squares method is an optimal estimation technique derived from the maximum similarity method when the random error is normal distribution [
21], which reduces the sum of distances between all points to be fitted and the fitting curve to its minimum. The general form of an ellipse is:
So the goal of the least squares method is to minimize the objective function f(A, B, C, D, E), which is defined as follows:
Then, the linear function is constructed through the extreme value theorem, and the ellipse parameters are solved by combining the constraint conditions. From the geometric properties of the ellipse, the parameters are:
Among them, (xe, ye) represents the center of the ellipse. (a, b) represent the major and minor axes of the ellipse, and φ is the inclination angle.
According to the principle of the least square method, too many outliers can lead to the random error not conforming to normal distribution, which will greatly affect the fitting effect.
2.3. A Defect Edge Repair Model and Evaluation Based on Approximate Circles
In the actual production and manufacturing of precision components, machining errors such as burrs may occur in the circular features to be measured due to processing methods, or significant errors may occur in the binary image results due to surface indentation. This can affect measurement accuracy during data fitting. The repair method for defective circles is mainly based on the principle that circles can be approximated as regular n-sided shapes. The schematic diagram is shown in
Figure 2.
Based on this issue, this article proposes a circular edge repair method based on the principle of circle approximation. It is based on the edge sequences of a circle obtained through edge detection after binarization, and connects the edge sequence points in a straight line to remove the protruding parts. Finally, the region with the largest connected domain is taken as the repaired feature area. The schematic diagram of edge repair is shown in
Figure 3.
Firstly, the method obtains an ordered edge sequence of features from the feature map that need to be repaired, and then traverses the edge sequence so that the traversed edge point is a (x
a, y
a), and its related point b (x
b, y
b) is the edge point with a sequence number after it, and the following conditions are met:
R is the repair radius. The edge repair algorithm is:
C(x
c, y
c) is the repair point, with its physical meaning being points a and b forming a single point on the line segment. We used the above method to process a regular hexagon, and included the Bresenham algorithm of computer graphics in the method’s implementation when cutting. The results are shown in
Figure 4.
From
Figure 4, it can be seen that this method is based on the approximation principle of circles. The cutting part is shown in red in
Figure 4b, and the repair result is shown in
Figure 4c. It can be seen that this method can effectively cut the features of the approximate circle into circles, that is, repair the shape of the approximate circle in circles. The same applies when detecting an ellipse. After multiple repairs, a smooth ellipse can be obtained, which can be used as an approximate circular target. Therefore, the limit on the number of repairs can be evaluated using circularity. The definition of circularity a is [
22]:
Among them, S is the area of the target and C is the circumference of the target. When the target shape is closer to a circle, its value is closer to 1. Therefore, whether the difference in roundness between the two repairs is less than the given threshold T is used to determine whether the repair is completed.
After obtaining the target that approximates a circle, a morphological dilation algorithm is used to process the repaired image. During each dilation process, the intersection of the repaired and original images’ edge points is recorded, and the part where the number of intersecting edge points increases due to each dilation is recorded. When the proportion of intersecting edge points exceeds the threshold, the dilation operation is stopped, and the edge point showing the largest increase in the number of intersecting points is taken as the effective edge point. By performing least squares ellipse fitting as described in the second section, the ellipse parameters can be obtained.
4. Conclusions
This article proposes a defect ellipse edge repair and fitting method based on vision for the rapid measurement of array micro oblique holes with defects such as scratches and burrs. Firstly, an edge repair algorithm is designed using the approximation principle of circles, and then the repair process is evaluated. Furthermore the repaired edges are obtained by firstly using morphological processing methods to obtain effective edge points of the ellipse to be fitted, which can effectively remove protruding outliers and concave outliers from the edge of the ellipse, and then use the least squares method is used to obtain effective ellipse parameters. This method is simple and effective in implementation, and can quickly obtain the parameters of defect ellipses. Through simulation and experiments, it can be determined that for oblique microwell images of 1920 × 1080 pixels, the dimensional variation measured by this method is less than 0.01 mm, and the angle error is less than 0.15°, which meets the accuracy requirements of an actual measurement. This method proposed in this article has a high measuring speed, which makes it suitable for engineering applications. In subsequent measurements, it can provide visual guidance for the precise measurement of fiber probes.