3.1. Stereovision Measurement Model
Since the image resolution varies with both the height and stagger values of the contact wire, the measurement of height and stagger is the basis of the measurement of contact wire wear. The measurement principle of the contact wire geometry parameters using the stereovision method is shown in
Figure 3. In the figure,
OwXwZw is the world coordinate system. The origin of this coordinate system,
Ow, is set at the midpoint of the train roof, and
Yw represents the direction of the train.
O1X1 and
O2X2 are the imaging coordinate systems of the left and right cameras, respectively.
Oc1Xc1Zc1 and
Oc2Xc2Zc2 are the camera coordinate systems of the left and right cameras, respectively. For these two camera coordinate systems, each origin is located at the principal point of the camera lens. The point
P is projected onto the imaging chip of the line-scan camera through the optical lens. The imaging locations of the point
P on the two cameras are
P1 and
P2, respectively.
U01 and
u02 are the image coordinates of the principal points of the two camera lenses.
By using perspective projection transformation, the transformation relation between the camera imaging coordinate system and the camera coordinate system is shown in Equation (3):
where
s is the non-zero scale factor;
is the coordinate of image point of point
P in any camera imaging coordinate system; [
Xc,
Zc] is the coordinate of point
P in the corresponding camera coordinate system; and
M1 is the internal parameter matrix of the camera and is given by the following:
In Equation (4), fe is the normalized focal length of the camera lens, and u0 is the image coordinate of the principal point of the camera lens.
By using Euclid-space transformation, the transformation relation between the camera coordinate system and the world coordinate system is as shown below [
22]:
where [
Xw,
Zw] is the coordinate of point
P in the world coordinate system;
M2 is the external parameter matrix of the camera; and
R and
T are the rotation matrix and translation vector, respectively.
Combining Equation (3) with (5), the line-scan camera model is given by the following:
where
M is the intrinsic parameter matrix.
M is described by the following:
By eliminating the variables in Equation (6), we can obtain the equation as follows:
Equation (8) is the perspective projection equation of the line-scan camera, which describes the relationship between the coordinate of point P in the world coordinate system and the image coordinate of point P in the camera imaging coordinate system.
Multiple sets of calibration point data are obtained within the measurement range, including the world coordinate data, [Xw, Zw], and the corresponding imaging coordinate data, u. Then, the parameters of the matrix, M, in Equation (8) are calculated using mathematical procedures.
In principle, the coordinate of point
P in the world coordinate system, [
Xw,
Zw], could be calculated by using the imaging coordinate data,
u, of any two line-scan cameras. The subscripts a and b are used to represent the left and right cameras in
Figure 3, respectively. By combining the perspective projection equations of the two cameras,
Xw and
Zw are given by the following:
The parameters
p1,
p2,
q1,
q2,
k1, and
k2 in Equation (9) are given by the following:
Equation (9) is the measurement model of the contact wire geometry parameters based on stereovision via triangulation. To perform the dynamic measurement, the pixel coordinates of the contact wire in the two line-scan cameras, ua and ub, are extracted through image processing. Then, the positions of the contact wire in the world coordinate system, [Xw, Zw], are calculated using Equation (9) and the calibrated parameters of the intrinsic parameter matrix, Ma and Mb, of the two line-scan cameras.
In the overlapping section [
1], not only the two contact wires, but also the messenger wires located above the contact wires, would be captured. As mentioned above, when there is one target in the FOV (field of view) of the line-scan camera, the world coordinate of the target could be calculated by using the imaging coordinate of the target in the images of two cameras. However, when the number of the targets is more than one, it is necessary to find the corresponding points of the same target between the stereo image pairs, that is, to solve the stereo-matching problem. Various research studies have been conducted to determine the accurate corresponding points, such as feature descriptors, interest point detectors, and epipolar constraint method [
14,
22].
This article proposes a matching method based on the position feature of the target. The third line-scan camera is adopted to overcome the correspondence problem. The target imaging coordinate information of the third line-scan camera is used to check the matching of corresponding points in the first two line-scan camera images, which could eliminate the uncertainty caused by the matching of binocular images.
The proposed matching method is illustrated in
Figure 4.
A,
B, and
C are three targets in the FOV of the stereovision system;
O1,
O2, and
O3 are the principal points of the three camera lenses;
a1,
b1, and
c1 are the imaging points of the three targets in the left camera;
a2,
b2, and
c2 are the imaging points of the three targets in the right camera; and
a3,
b3, and
c3 are the imaging points of the three targets in the middle camera. The matching procedure operates in three steps, which are described as follows.
Step 1: All possible spatial points are reconstructed based on the extracted image coordinate data of the left and the right cameras, using the enumeration method. Among the possible spatial points shown in
Figure 4,
A,
B, and
C are the real targets, while
D,
E, and
F are the false targets. Then, the key to the matching procedure is to verify all of the reconstructed spatial points.
The verification of points A and D is illustrated bellow. Point A is reconstructed using the imaging point a1 of the left camera and a2 of the right camera. Point D is reconstructed using b1 of the left camera and a2 of the right camera.
Step 2: The reconstructed points A and D are re-projected onto the imaging chip of the middle line-scan camera and form the re-projected imaging points a3p and d3p. The re-projection imaging coordinates of points a3p and d3p are denoted as ua3p and ud3p, respectively.
Step 3: The imaging coordinates of a3, b3, and c3 are denoted by ua3, ub3, and uc3. The validity of the re-projected imaging points a3p and d3p is verified by calculating the distance between the extracted image coordinate data and the re-projected imaging coordinate data. An appropriate threshold, T, is adopted by taking into account the average deviation distance of re-projection of the line-scan camera. With regard to the re-projected imaging point a3p, the distance between ua3 and ua3p is less than the threshold (T), verifying the effectiveness of reconstructed point A. However, reconstructed point D is judged as a false target since there is no candidate imaging point near the re-projected imaging point d3p.
The verification-procedure speed in this study is fast and satisfies the requirement of real-time measurement. In addition, the matching method is robust for both day and night inspection since the matching precision is not easily affected by the changes in ambient lighting.
3.2. Determining the Wear Width of the Contact Wire
Determining the wear width of the contact wire includes two steps, the extraction of wear width in pixels from the image and the calculation of the physical wear width in mm. The specific steps of extracting the wear width in pixels from the image are as shown below:
Step 1: The candidate wires are extracted by using the edge detection operator. The first-order difference of grayscale curve is designed to detect the candidate objects in different weather conditions.
Figure 5 shows the original grayscale curve of catenary on a cloudy day in an overlapping section. Therefore, there are two peak-shaped regions and two valley-shaped regions in the background grayscale curve, where the peak-shaped regions are the two contact wires in the overlapping section and the valley-shaped regions are the two messenger wires.
Figure 6 shows the processed images by using the first-order difference operator. The grayscale gradient of the contact wire is obviously higher than that of the messenger wire, which helps to minimize the interference from the messenger wire in matching.
Step 2: The peak-shaped regions and the valley-shaped regions are extracted from the fitting background grayscale curve. Mean filter operator is adopted to acquire the fitting background grayscale curve and minimize the influence of noise. Then, the boundary between the contact wire and the background is carefully determined. An empirical threshold of the pixel width of a contact wire is used to assist in determining boundaries. The threshold is determined by experimental statistics, and the value of the threshold is affected by the resolution of the line-scan camera, the FOV of the camera and the height variation in the contact wire.
Step 3: The left and right edge points of the wear surface are located within the peak-shaped region of a contact wire in the image. As mentioned above, the wearing surface has much higher light-reflecting characteristics than the lateral surface of the contact wire. As a result, the grayscale of the wear surface region is several times higher than that of the rest of regions of the contact wire. As shown in
Figure 7, the grayscale of the lateral surface regions of the contact wire decreases rapidly from the wear edge point to the background. In this case, the second-order difference of the grayscale curve is adopted to locate the left and right wear edge points within the peak-shaped region of the local grayscale curve of the contact wire. The wear width in pixels is extracted by calculating the difference of the coordinates between the left and the right wear edge points. Meanwhile, the central point of the wear surface is determined.
Step 4: The real targets in the FOV are determined by using the stereovision matching method proposed in this study. Several targets may be extracted in an overlapping section in daytime inspection, i.e., the operating contact wire, the non-operating contact wire, and the two messenger wires which support these two contact wires. The position of each target in the world coordinate system is calculated by triangulation, using Equation (9).
Step 5: The messenger wires are excluded based on the fact that the height of the contact wire is lower than that of the messenger wire. A target-tracking operator is used to exclude the other interference targets, such as the droppers and the cantilevers of the support system. This is due to the fact that the contact wire is continuous along the railway line, while the projections of both the droppers and the cantilevers on the image are discontinuous in the direction of the railway line.
The calculation of the physical wear width in mm relies on the determination of the image resolution in the measurement range. For different types of electrified railways in China, the height from the car roof to the contact wire would be in the range between 1300 mm and 2500 mm. The stagger would vary from −400 mm to 400 mm in an electrified railway to avoid the continuous friction at the same point of the pantograph. If the effect of the stagger variation is not taken into account, a significant error would be introduced when considering the image resolution. In this study, the image resolution in various sections of measurement range is carefully calibrated using a calibration tool, which takes into account the impact of both the height variation and the stagger variation. As shown in
Figure 8a, a set of horizontally arranged targets is fixed on top of a slide bar, with a spacing of 100 mm between each target. A flat surface with a width of 6 mm is fabricated on the bottom of each target, which simulates a contact wire with a wear surface, as shown in
Figure 8b. Each target in the horizontal array denotes a contact wire with a different stagger value. The height of the target array could be set in the measurement range by moving the slider bar up and down.
First, the parameters of the matrix (
M) in Equation (8) of each line-scan camera are calculated using multiple sets of calibration data, including the world coordinate data, [
Xw,
Zw], and the corresponding imaging coordinate data,
u. After that, the wear width of each target is extracted in pixels from the captured grayscale curve, using the above image-processing procedure, as shown in
Figure 9.
The image resolution with a fixed position is calculated by dividing the preset physical width of 6 mm by the extracted image width of the wear surface, with the unit of mm/pixel.
Table 1 illustrates the calibrated image resolutions of the second line-scan camera at different horizontal positions with a height of 1300 mm from the car roof. The image resolutions of each line-scan camera at different horizontal or vertical positions are determined by repeating the above calibration. Thus, the measurement range is divided into an array of small square areas. The image resolution within the same small square area is considered to be uniform. As a result, the distribution of the image resolution in the measurement range is determined by using this lookup-table method. Then, the physical wear width is calculated accordingly. Finally, the residual thickness,
h, of the contact wire is calculated according to the wire type and the rated cross-section diameter of the contact wire by using Equation (1).
3.3. Measurement Apparatus Design
The architecture diagram of the measurement apparatus based on the measurement method for contact wire wear in this study is shown in
Figure 10. The measurement apparatus is composed of the stereovision measurement module on the train roof, the processing module inside the train, and the vehicle compensation measurement module under the train. The stereovision measurement module consists of four line-scan cameras with different angles and three LED lamps. The vehicle compensation measurement module includes three displacement sensors. When the train sways, two sensors are used to measure both the left and the right vertical displacement of the train relative to the rail surface, and then the roll angle of the train is calculated by using the displacement data of these two sensors and the width of the train; the third sensor is applied to measure the horizontal displacement of the train relative to the center line of the track. The processing module processes the image data of catenary captured by the line-scan cameras and the vehicle compensation data and calculates the geometry parameters (stagger and contact wire height) and the wear parameters (residual thickness,
h; or worn area,
A). The processing module receives the distance pulses produced by the photoelectric encoder, which is installed on the train wheel and triggers all the line-scan cameras to exposure simultaneously.
The flowchart of the processing module to perform the measurement of the contact wire wear is shown in
Figure 11, which includes four steps:
Step 1: Acquire the image data of the four line-scan cameras and the voltage data of the three displacement sensors.
Step 2: Extract the features from each line-scan camera’s image, including the central point coordinate and the wear width of the candidate targets.
Step 3: Execute the stereovision matching algorithm to determine the corresponding points of the contact wire among the stereo images, and then calculate the position of the contact wire in the train roof coordinate system.
Step 4: Calculate the position of the contact wire in the rail surface coordinate system by using the calculated position data from Step 3 and the measurement data of the vehicle compensation measurement module based on the Euclid-space transformation. Then, the physical wear width of the contact wire is determined by using both the data of the wear width in pixels from Step 2 and the image resolution data corresponding to the current position.
The catenary image quality is essential to the wear measurement of the contact wire. In this study, a new kind of high-speed synchronized stroboscopic lighting technology is developed as an alternative active light source. The high-speed synchronization of lighting and camera exposure is realized by a same-trigger pulse. The imaging and lighting devices are able to work stably at 1000 Hz in the pulsed mode, which ensures the dynamic measurement of contact wire wear on the inspection train. Since the duty cycle of the pulsed lighting is low (less than 0.1), the power consumption of the light source is greatly reduced, which is no more than 5% of the spotlights’ total power in our previous research [
19,
20]. Compared with the continuous LED lighting, the thermal performance of the LED chip in stroboscopic mode is significantly improved, which greatly extends the lifetime of the light source. Moreover, compared with the continuous intensity lighting with rated current, an over-driven pulsed current is used to obtain a light beam with higher luminous flux, which highlights the image features of the wear surface of the contact wire.
Blue LED lamps combined with band-pass optical filters are adopted to minimize the influence of sunlight. A comparison test in daytime was performed on two light sources, that is, the white LED lamp and the blue stroboscopic LED lamp with the optical filter, as shown in
Figure 12. In
Figure 12a, the light zigzag lines are the contact wires due to the good light-reflecting characteristics, while the dark zigzag lines are the messenger wires due to the poor reflectivity. In
Figure 12b, the grayscale of the sky is greatly reduced, and the image SNR in the daytime environment is significantly improved by using the proposed lighting technology. The messenger wires disappear in
Figure 12b because of the little difference in grayscale between the messenger wires and the sky. In this case, the number of the extracted candidate targets is reduced, which accelerates the stereo matching.