1. Introduction
Cranes are important equipment for the development of modern industry and are widely used in ports, metallurgy, urban construction, aerospace, petrochemical and other fields. The role of the crane in the process of cargo transportation is irreplaceable as it can transport high-load cargo, such as the hoisting of offshore drilling platforms, the assembly of heavy ships, and the hoisting of nuclear power plant containment domes [
1,
2,
3,
4,
5].
With the continuous development of hydraulic technology [
6], computer technology [
7], communication technology [
8], advanced control technology [
9], and new energy technology [
10,
11,
12], the intelligence level of cranes is also constantly improving, but its structural safety [
13] is always an important part of its safety. Once there is a problem with the structural safety of large construction machinery, it is easy to cause significant loss of life and property [
14]. Therefore, large-scale construction machinery such as hoisting machinery is also a highly dangerous mechanical equipment [
15].
In the past 20 years, safety assessment has been gradually applied in structural engineering, chemical disasters, information security, and other fields, while there have been relatively few safety assessment theories for cranes [
16,
17,
18]. The load of the crane is usually large, ranging from several tons to thousands of tons, and the lifting height ranges from tens of meters to hundreds of meters [
19]. Therefore, with the increasing application of cranes, the number of accidents is also rising. Therefore, it is very important to carry out safety assessment and risk monitoring of cranes.
The structural safety assessment of cranes refers to the assessment of the potential hazards and severity of mechanical structures. Commonly used crane risk assessment methods include fuzzy comprehensive assessment method, risk assessment method based on combination weighting, comprehensive risk assessment method, etc. [
20]. But no matter which risk assessment method is used, key dimensions or the coordinates of key points need to be obtained. For construction machinery in complex working environments, truss is a common local structure. The key points on the crane usually show a face on the image, so the selection of measuring tools is particularly critical. Especially for the complex local structure of the crane, the design scheme of the engineering test is also critical, which is related to whether the key point coordinates can be accurately measured. After the key points are measured, they can be used as the basic data for crane safety evaluation. These data will play an important role in crane safety evaluation.
In the field of engineering, the commonly used measurement methods are: coordinate measuring machine, articulated arm measuring machine, laser tracker, structured light measurement system, total station, laser scanner, photogrammetry, etc. [
21]. The three-coordinate measuring machine (CMM) has high measurement accuracy, good flexibility, and strong reverse engineering capabilities [
22]. It is widely used in the mold industry, and it is a modern intelligent tool that integrates design, testing, and statistical analysis. As a portable measuring device, the articulated arm measuring machine needs to touch the point to be measured in space to complete the measurement [
23]. The laser tracker uses a spherical coordinate system and relies on single-frequency laser interferometric ranging, which has high measurement accuracy and fast measurement speed. Moreover, laser tracker has certain advantages in large scenes [
23]. Structured light measurement systems are divided into line structured light measurement systems and surface structured light measurement systems, which use the principle of triangulation to obtain the three-dimensional coordinates of points [
24]. The total station is a high-precision measurement device that integrates light, machinery, and electricity. Its basic measurement principle is the same as that of the laser tracker, but the distance measurement method is different. Its measurement distance is long and its application range is very wide [
25]. Laser scanners use a large number of laser points to form point clouds, so that three-dimensional information of the outer surface of the object to be measured can be obtained [
26]. Photogrammetry is based on the relative positional relationship between the photographic center, image point, and spatial point, using two or more images to complete the coordinates calculation of the spatial points, and the measurement distance can be close or far [
27]. Among the above measurement methods, the CMM, the articulated arm measurement machine, and the structured light measurement system have shorter measurement distances. Both the articulated arm measuring machine and the laser tracker are contact measuring methods. The operation of the total station is complicated, and the coordinates of a large number of points must be measured one by one. Laser scanners are extremely expensive to purchase and expensive to maintain. None of these measurement methods can meet the measurement needs of large construction machinery such as port machinery. The structure of large construction machinery such as port machinery is usually relatively tall, the range to be measured is relatively wide, and its working environment is complex. These factors determine that the measurement method that can be widely used in port machinery and other large construction machinery must be efficient, non-contact, portable, and have long measuring distance. Among the existing measurement methods, photogrammetry is undoubtedly the most advantageous. With the development of computer technology and artificial intelligence technology, the accuracy and stability of image-based visual measurement are also constantly improving. The intersection of close-range photogrammetry and advanced technologies such as computer vision is getting deeper and deeper, so the application range of close-range photogrammetry is also expanding [
28].
The demand for close-up photogrammetry in aerospace [
29], automobile manufacturing [
30], mold manufacturing [
31], material science [
32], biomedicine [
33], cultural heritage protection [
34], etc. is increasing. After close-range photogrammetry entered the digital stage in 1980, many digital close-range photogrammetry systems appeared. There are TRITOP system of German GOM company, DPA-Pro system of German AICON 3D company, etc., whose technologies all come from the IAPG Institute. In addition, there are the V-STARS system of GSI Corporation of the United States, the Australis system of Photometrix Corporation of Australia, and the Metronor system of Metronor Corporation of Norway. The V-STARS system was developed by GSI Corporation of the United States and is currently the most mature commercial digital close-range industrial photogrammetry system in the world [
35]. The Lensphoto multi-baseline digital close-up photogrammetry system, which was directly participated and launched by Academician Zhang Zuxun of Wuhan University, has been applied in the fields of water conservancy and electric power measurement, building measurement, cultural relics protection, and other fields [
36].
2. Related Work
The application of close-range photogrammetry in port machinery and other large construction machinery is also increasing. Lu Enshun from Wuhan University of Technology [
37] developed a photogrammetry-based radius measurement algorithm for the structural characteristics of port machinery, he used the approximate distortion model of the camera to determine the weight of each midperpendicular, and compared the equal-weight fitting algorithm with the weighted fitting algorithm. Wang Qi [
38] developed a focal length calibration algorithm based on photogrammetry for port cranes. The central idea is to obtain image information through the Internet of Things technology, and then obtain the attitude parameters by minimizing the error function. Lin Xuanxiang [
39] disclosed a method for detecting the deformation of the main beam of a hoisting machine based on photogrammetry, which uses photogrammetry to obtain the coordinates of the point to be measured, and then fits a curve according to the obtained coordinates to determine the deformation of the main beam. At present, the application of photogrammetry in large-scale construction machinery is relatively small, and there is a relatively broad research space.
In the theory of close-range photogrammetry [
40], there are usually observation errors, so redundant observations are introduced to increase the accuracy of the calculation. Using multiple images to complete the space intersection is a commonly used adjustment method. Li Zhongmei added the overall least squares estimation in the process of multi-image space intersection and removed the gross error. Li Jiatian [
41] combined the space intersection with the neural network to reduce the influence of nonlinear errors on solving three-dimensional coordinates. Faugeras and Mourrain [
42] creatively gave a new derivation of the three-focal tensor equation, using three images to complete the solution; this method has been widely used in various fields. In the optimal solution triangulation of three views, Stewenius et al. used the method of interactive algebra to solve the polynomial matrix to obtain the triangulation result [
43]. Agarwal et al. addressed the problem of global triangulation using fractional programming methods [
44]. Dai et al. proposed a norm-based optimization method to improve the efficiency of triangulation [
45]. In the multi-view triangulation, Zhang et al. selected some observation vectors as subsets, and then solved the subset data, which also improved the efficiency of triangulation [
46].
Entropy is a thermodynamic concept proposed by physicist R. Clausius. C.E.Shannon first introduced entropy in thermodynamics into information theory. The appearance of information entropy is a sign of the generalization of entropy. It is widely used in physics, chemistry, medicine, water conservancy, communication, and mechanical safety assessment [
47]. Since information entropy can measure the uncertainty of the appearance of things, it is also widely used in image processing to increase the accuracy of feature extraction, but there are few studies that can combine the basic principles of information entropy and photogrammetry.
For the safety assessment of large machinery such as port machinery, critical dimensions or the coordinates of key points are very important. Due to the limitation of the shooting angle or the occlusion of other mechanical structures, the key points are likely to be occluded or cannot be directly measured. Inspired by previous studies by scholars, the key points for dimension measurement or safety assessment are generally the intersection points of three straight lines or corner points with obvious features [
48,
49]. Combining structural features of large construction machinery, a prediction and optimization algorithm for the intersection point of multiple space straight lines is proposed [
41,
50,
51]. In order to comprehensively consider the distortion of image points [
52,
53], the error of point selection, and the different shooting conditions of each image, the algorithm introduces information entropy [
54,
55]. Then the spatial lines involved in the calculation of fitting points are weighted [
37] to further improve the fitting accuracy.
Therefore, compared with the previous work, the innovations of this paper are as follows: (1) we use multiple images to predict and optimize the intersection of straight lines; (2) an iterative optimization method based on reprojection is proposed; and (3) we introduce information entropy and weight the space line.
The rest of this paper is arranged as follows.
Section 3.1 introduces the intersection point prediction algorithm of equal-weighted multiple lines.
Section 3.2 introduces the intersection point prediction algorithm of weighted multiple lines.
Section 4 introduces three experiments, all of which are used to verify the accuracy and stability of the algorithm in this paper.
Section 5 provides a detailed and comprehensive analysis of the experimental results.
Section 6 summarizes the entire paper.
6. Conclusions
Based on the basic theory of photogrammetry and the structural characteristics of large engineering machinery such as port machinery, a prediction and optimization algorithm for intersection point of spatial multi-Lines based on photogrammetry is proposed under the condition of considering the influence of many factors such as image point distortion and point selection error. The algorithm takes the spatial fitting point calculated by the traditional method as the initial point of weighted iteration. The projection points on each image are obtained from the spatial fitting points, and the distance between the projection points and each line in the image plane is combined with the information entropy to determine the weight of the space line, and then more accurate spatial coordinates of the intersection points can be obtained. Experimental results show that the proposed iterative optimization algorithm based on information entropy can significantly improve the accuracy of fitting points, and this algorithm has strong practicability.
Due to the limitations of experimental conditions and in order to better present comprehensive experimental data, the experiment in this paper only uses three images to calculate the intersection point. In subsequent studies, more images can be used to solve the problem to obtain more accurate fitting point coordinates. This method is not only applicable to port machinery, but also to other large engineering machinery. At the same time, this paper also introduces the information entropy in the proposed algorithm and uses information entropy to determine the relative weights. In future studies, we will focus on other structural characteristics of port machinery and other large machinery, and more a targeted algorithm will be put forward. We will make the photogrammetry applied more widely in engineering, at the same time, we will try to apply information entropy to more aspects of photogrammetry.