High-Precision Drilling by Anchor-Drilling Robot Based on Hybrid Visual Servo Control in Coal Mine

Abstract

Rock bolting is a commonly used method for stabilizing the surrounding rock in coal-mine roadways. It involves installing rock bolts after drilling, which penetrate unstable rock layers, binding loose rocks together, enhancing the stability of the surrounding rock, and controlling its deformation. Although recent progress in drilling and anchoring equipment has significantly enhanced the efficiency of roof support in coal mines and improved safety measures, how to deal with drilling rigs’ misalignment with the through-hole center remains a big issue, which may potentially compromise the quality of drilling and consequently affect the effectiveness of bolt support or even result in failure. To address this challenge, this article presents a robotic teleoperation system alongside a hybrid visual servo control strategy. Addressing the demand for high precision and efficiency in aligning the drilling rigs with the center of the drilling hole, a hybrid control strategy is introduced combining position-based and image-based visual servo control. The former facilitates an effective approach to the target area, while the latter ensures high-precision alignment with the center of the drilling hole. The robot teleoperation system employs the binocular vision measurement system to accurately determine the position and orientation of the drilling-hole center, which serves as the designated target position for the drilling rig. Leveraging the displacement and angle sensor information installed on each joint of the manipulator, the system utilizes the kinematic model of the manipulator to compute the spatial position of the end-effector. It dynamically adjusts the spatial pose of the end-effector in real time, aligning it with the target position relative to its current location. Additionally, it utilizes monocular vision information to fine-tune the movement speed and direction of the end-effector, ensuring rapid and precise alignment with the target drilling-hole center. Experimental results demonstrate that this method can control the maximum alignment error within 7 mm, significantly enhancing the alignment accuracy compared to manual control. Compared with the manual control method, the average error of this method is reduced by 41.2%, and the average duration is reduced by 4.3 s. This study paves a new path for high-precision drilling and anchoring of tunnel roofs, thereby improving the quality and efficiency of roof support while mitigating the challenges associated with significant errors and compromised safety during manual control processes.

1. Introduction

Bolt support is currently the primary method for the reinforcement of coal-mine roadways in China. Approximately 12,000 km of coal-mine roadways are excavated annually in China, with 80% of them supported by rock bolts [1]. This method effectively enhances the stability of roadway-surrounding rocks, reducing the occurrence of geological hazards such as rock collapses and landslides. Simultaneously, it mitigates roadway deformation and roof collapses, thereby providing a safer working environment for miners [2,3]. Presently, the installation of roof rock bolt support in roadways primarily relies on manual operation of individual equipment by workers, posing challenges including slow support speed, inadequate effectiveness, high labor intensity, and poor safety.

In response to the inefficiency of traditional rock bolt support methods, domestic and international research institutions have undertaken thorough studies and technological innovations, resulting in the design and development of various new integrated drilling and anchoring equipment. These equipment have significantly enhanced the level of mechanization, thereby effectively improving the efficiency of roof rock bolt support in coal-mine roadways. Among them, the ABM20 series integrated drilling and anchoring machine, the MB670 anchoring machine by Sandvik, the 12CM series integrated drilling and anchoring machine by JOY, and the tunnel support mechanical arm M2000-EBH developed by GTA corporation all demonstrate significant advantages in synchronized drilling and anchoring operations. Recent advancements in this equipment have significantly contributed to labor reduction and efficiency improvement in coal-mine tunnels. These new drilling and anchoring equipment enable simultaneous tunnel excavation and roof support, thus avoiding the time-consuming issue of frequent machine relocations. Their higher integration level allows for increased mechanization, resulting in a marked improvement in support efficiency compared to traditional methods. Most of these new equipment include temporary support devices, and workers can operate them from within a control room, thereby avoiding the risks of working under unsupported rock and reducing safety incidents. However, it is worth noting that despite the outstanding performance of these new integrated drilling and anchoring machines in coal mines with favorable geological conditions, their application still faces challenges under certain complex geological conditions. As shown in Figure 1, the current main process for rock bolt support includes placing the steel strip on a temporary support device, with evenly distributed circular holes on the steel strip serving as drilling holes; controlling the drilling rig to drill through the drilling holes in the tunnel roof, where the center of the drilling hole is the designed drilling position; and, after completing the drilling, retracting the drill rod, replacing it with a rock bolt, and performing the drilling operation to ensure that the rock bolt is securely fixed in the drilled hole. The steel strip provides positioning and stabilization during the drilling process to ensure drilling accuracy, and during the drilling process, it works together with the rock bolt to prevent rock layer collapse and enhance support effectiveness. This support process can significantly improve the stability and safety of the tunnel roof. However, if the drilling rig cannot accurately align with the center of the drilling hole, it will not be able to drill at the designed roof position and may instead drill into the steel strip, making it impossible to complete subsequent tasks and posing certain dangers. Since workers control the drilling rig from a control room, the rapid and accurate alignment of the drilling rig and the center of the drilling hole is still a prominent challenge, which leads to considerable alignment errors.

Robotic manipulators are highly favored by both academia and industry due to their efficient operational capabilities and extensive application prospects [4,5,6]. Therefore, we integrated a robotic manipulator with a roadheader to construct an anchor-drilling robot, whose end-effector is a drilling rig, used for drilling and anchoring. The anchor-drilling robot is widely used in anchor drilling in coal-mine roadways, which significantly improves the efficiency and safety of support. This system accomplishes the reinforcement of coal-mine roofs, thereby averting collapse accidents.

With the rapid development of computer vision, it has been widely used in target recognition [7,8], defect detection [9,10], automatic driving [11,12], and medical diagnosis [13,14]. Combining computer vision technology with robot technology can realize specific tasks, which are the so-called visual servo control methods. Visual servo control methods can be categorized into position-based visual servo (PBVS), image-based visual servo (IBVS), and hybrid visual servo (HVS) control methods based on different feedback signals.

PBVS control methods have found extensive applications in high-precision tasks such as part assembly [15,16] and robot navigation [17,18] due to their simple structure and ease of implementation. However, the control precision of PBVS is highly contingent upon the calibration accuracy of both the camera and the robot, exhibiting extreme sensitivity to minute errors in calibration parameters [19,20]. Moreover, the accurate estimation of the spatial pose of the target object significantly impacts control precision. Therefore, in the design of PBVS systems, particular attention must be paid to the accuracy of the calibration process and the robustness of spatial pose estimation algorithms to ensure system stability and reliability.

To mitigate the adverse impact of calibration errors on the precision of visual servo control, researchers often favor the adoption of IBVS methods [21,22,23]. IBVS directly utilizes features extracted from images to generate control signals, offering an intuitive control strategy with lower sensitivity to calibration errors. In IBVS, the precise computation of the image Jacobian matrix is pivotal, as it delineates the dynamic relationship between changes in image features and robot motion, serving as a critical bridge for mapping from the image feature space to the robot’s workspace. However, the accurate computation of the image Jacobian matrix poses a technical challenge in IBVS methods. Presently, analytical methods [24,25] and neural network approaches [26,27] are primarily employed for estimating the image Jacobian matrix. While analytical methods are simple, they are constrained by linear systems and may not yield satisfactory results for nonlinear systems. Conversely, neural network methods rely on the quantity and quality of training data, with the quality of computational results directly influenced by dataset characteristics. For addressing complex nonlinear problems, the particle swarm optimization (PSO) algorithm [28,29], as a global optimization method, demonstrates potential application value in the estimation of the image Jacobian matrix.

Moreover, HVS [30,31] methods, as a fusion of PBVS and IBVS, aim to achieve higher control precision and robustness. However, HVS necessitates the handling of multiple information sources and their integration within a unified framework, leading to higher computational complexity compared to singular PBVS or IBVS methods. To meet real-time requirements, implementing HVS methods may entail higher-performance computing resources. Thus, optimizing HVS algorithms to reduce computational costs while maintaining control performance is one of the current research’s vital directions, particularly in resource-constrained environments.

Therefore, to meet the requirement of rapid and precise alignment of the drilling rig with the center of the target drilling hole, a hybrid visual servo control strategy combining PBVS and IBVS is proposed. Firstly, a binocular vision measurement system is employed to determine the position of the drilling-hole center, establishing a PBVS control system to guide the drilling rig effectively toward the center of the target drilling hole. When the distance between the drilling rig and the target drilling hole decreases below a preset threshold, the system switches to a monocular vision system. Utilizing the image information collected by this system, the vertex coordinates of the bounding rectangle of the drilling hole are used as feature points to construct an IBVS control system, facilitating the rapid and precise alignment of the drilling rig with the center of the drilling hole. By leveraging the advantages of both visual servo control strategies, the proposed hybrid control strategy achieves rapid and precise alignment of the drilling rig with the center of the drilling hole, thereby completing the drilling and anchoring tasks.

In summary, the main contributions of this study are as follows:

(1)

A method for controlling the drilling rig based on a hybrid visual servo is proposed, aiming to achieve rapid and precise alignment of the drilling rig with the center of the target drilling hole;

(2)

A method for estimating the image Jacobian matrix based on the PSO algorithm is presented, enabling rapid and accurate control of the drilling rig through the IBVS method;

(3)

Simulation and experimental results demonstrate the substantial benefits of the proposed method. The findings indicate that this approach significantly improves the efficiency of roof support in coal-mine tunnels.

The remainder of this paper is organized as follows: Section 2 introduces the problem description and the model of the anchor-drilling robot. The hybrid visual servo methods are presented in Section 3. Section 4 presents the simulation and experimental results. Finally, Section 5 concludes the article.

2. Preliminaries

2.1. Task Description

During the excavation of coal-mine roadways, the stress levels of the encompassing rock strata are altered significantly. This typically results in roof-fall accidents, which consequently adversely affect the safety of mining operations. Currently, the primary strategy employed to maintain the stability of the rock surrounding the roof is bolt support, which encompasses both drilling and anchoring processes. Parallel to the rapid advancement in the automation of coal mines, the task of supporting the roof in coal-mine roadways has transitioned from manual labor to the utilization of anchor-drilling robots. This shift not only enhances the efficiency of support operations but also significantly diminishes the physical exertion required by the workforce.

A structural diagram of an anchor-drilling robot, which integrates a roadheader and two six-degree-of-freedom manipulators, is shown in Figure 2. The roadheader is designed for excavating coal-mine roadways. In contrast, the two six-degree-of-freedom manipulators are specifically designed to execute drilling and anchoring tasks on the roof of roadways. These manipulators comprise four rotational joints and two translational joints, enabling them to carry out their designated functions efficiently. Drilling rigs are rigidly attached to the extremities of these manipulators and function as end-effectors. The primary focus of this study is to identify a methodological approach for the rapid and precise alignment of the drilling rig with the drilling-hole center.

The anchor-drilling robot is equipped with visual sensors to rapidly and accurately align the drilling rig with the drilling-hole center. To establish a hybrid visual servo control model, the anchor-drilling robot is equipped with a set of binocular vision systems and a set of monocular vision systems. Specifically, the binocular vision system is installed on the body of the anchor-drilling robot, primarily serving to achieve precise positioning of the target drilling-hole center. This system is instrumental in constructing a PBVS control model, guiding the robot toward the target drilling hole to ensure the accuracy of the initial alignment. Conversely, the monocular vision system is mounted at the end of the drilling rig, responsible for capturing real-time image information. When the drilling rig approaches the target drilling hole within a preset threshold, the image information is utilized to construct an IBVS control model. This model aims to achieve rapid and precise alignment between the drilling rig and the drilling-hole center, ensuring the efficiency and accuracy of the drilling and anchoring operation.

To automate the alignment process, four coordinate systems are defined as follows: the manipulator base coordinate system $O_B{X}_B{Y}_B{Z}_B$, the end-effector coordinate system $O_E{X}_E{Y}_E{Z}_E$, the binocular coordinate system $O_{C1}{X}_{C1}{Y}_{C1}{Z}_{C1}$, and the monocular coordinate system $O_{C2}{X}_{C2}{Y}_{C2}{Z}_{C2}$. Simplified diagrams of the different coordinate systems are shown in Figure 2. Equipped with corresponding angle sensors or displacement sensors for each joint of the manipulator, the rotation angle or moving distances of each joint can be monitored in real-time. Using the coordinate transformation relationships helps calculate the position and attitude of the drilling rig relative to the base coordinate system of the manipulator in real time.

2.2. Solution Methodology

The framework proposed herein, which is highly systematic and innovative, is shown in Figure 3. The core of this method is the application of an advanced vision system to anchor-drilling robots. The binocular vision system identifies and determines the spatial position of the drilling hole by capturing an image of the roof steel strip. Specifically, the system uses circular holes on the steel stripe as positioning indicators for drilling holes, identifies these circular holes via image-processing technology, and calculates the accurate spatial position of the drilling holes. Subsequently, a PBVS control strategy is adopted to control the manipulator as it approaches the drilling hole.

When the manipulator approaches the drilling hole, and the distance from the center of the hole is reduced to less than 10 cm, the operation is considered fully completed. Subsequently, the monocular system on the drilling rig is operated, the image of the drilling hole is captured, and the four vertices of the hole’s circumscribed rectangle are regarded as the key feature points. Based on these feature points, an efficient IBVS control model was constructed. This model enables a fast and accurate alignment of the anchor-drilling center, thus effectively completing the drilling and anchoring tasks. PBVS control enables the coarse control of the manipulator, whereas IBVS control achieves precise manipulation. The method developed in this study not only improves work efficiency but also ensures the accuracy and safety of the task.

2.3. Kinematic Analysis of the Manipulator

Considering the identical structural configuration of the left and right manipulators in the anchor-drilling robot, this study focuses on one of the manipulators for a detailed examination. The construction of the manipulator model is refined using the modified Denavit–Hartenberg (MDH) parameter method, as shown in Figure 4. Within this framework, the coordinate system $x_i{y}_i{z}_i$ is established for each joint, where the coordinate system $x_0{y}_0{z}_0$ coincides with and is fixed to the coordinate system $x_1{y}_1{z}_1$ on the base. $d_3$ is defined as the offset for connecting rod 3, and $d_6$ signifies the offset for connecting rod 6. ${\theta }_1$, ${\theta }_2$, ${\theta }_4$, and ${\theta }_5$ denote the rotational angles of the corresponding joints.

According to the established coordinate system of the connecting rod and the structural parameters of the manipulator, DH parameters are shown in

Table 1. DH parameter table of the manipulator of anchor-drilling robot.

i	αi−1/(°)	ai−1/(mm)	di/(mm)	θi/(°)
1	0	0	0	θ1
2	90	0	0	θ2 + 90
3	90	0	1200 + d3	180
4	90	0	0	θ4 + 180
5	90	200	400	θ5 + 90
6	90	0	200 + d6	0

, where i represents the joint, α_i−1 represents the torsion angle of the connecting rod, a_i−1 represents the length of the connecting rod, d_i represents the offset of the connecting rod, and θ_i represents the joint angle.

The transformational relationship between the coordinate system {i} and its predecessor {i − 1} is mathematically articulated through a transformation matrix.

$${}_{\mathrm{i}}{}^{i-1}T=\left[\begin{array}{cccc}\cos {\theta }_i& -\sin {\theta }_i& 0& a_{i-1}\\ \sin {\theta }_i\cos {\alpha }_{i-1}& \cos {\theta }_i\cos {\alpha }_{i-1}& -\sin {\alpha }_{i-1}& -\sin {\alpha }_{i-1}{d}_i\\ \sin {\theta }_i\sin {\alpha }_{i-1}& \cos {\theta }_i\sin {\alpha }_{i-1}& \cos {\alpha }_{i-1}& \cos {\alpha }_{i-1}{d}_i\\ 0& 0& 0& 1\end{array}\right]$$

(1)

Consequently, the transformation matrix of the six-degree-of-freedom manipulator’s end-effector coordinate system relative to the base coordinate system of the manipulator is as follows:

$${}_6{}^{0}T={}_1{}^{0}T{}_2{}^{1}T{}_3{}^{2}T{}_4{}^{3}T{}_5{}^{4}T{}_6{}^{5}T=\left[\begin{array}{cccc}{n}_x& o_x& a_x& p_x\\ n_y& o_y& a_y& p_y\\ n_z& o_z& a_z& p_z\\ 0& 0& 0& 1\end{array}\right]$$

(2)

The elements of the transformation matrix are as follows:

$$\left\{\begin{array}{cc}\hfill n_x& =\cos {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\sin {\theta }_5+\sin {\theta }_1\cos {\theta }_5\hfill \\ \hfill o_x& =\cos {\theta }_1\cos \left({\theta }_2+{\theta }_4\right)\hfill \\ \hfill a_x& =-\cos {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\cos {\theta }_5+\sin {\theta }_1\sin {\theta }_5\hfill \\ \hfill p_x& =-\left(\cos {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\cos {\theta }_5-\sin {\theta }_1\sin {\theta }_5\right)\left(200+d_6\right)\hfill \\ \hfill & +400\cos {\theta }_1\cos \left({\theta }_2+{\theta }_4\right)-200\cos {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\hfill \\ \hfill & +\cos {\theta }_1\cos {\theta }_2\left(1200+d_3\right)\hfill \\ \hfill n_y& =\sin {\theta }_1\cos \left({\theta }_2+{\theta }_4\right)\sin {\theta }_5-\cos {\theta }_1\cos {\theta }_5\hfill \\ \hfill o_y& =\sin {\theta }_1\cos \left({\theta }_2+{\theta }_4\right)\hfill \\ \hfill a_y& =-\sin {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\cos {\theta }_5-\cos {\theta }_1\sin {\theta }_5\hfill \\ \hfill p_y& =-\left(\sin {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\cos {\theta }_5+\cos {\theta }_1\sin {\theta }_5\right)\left(200+d_6\right)\hfill \\ \hfill & +400\sin {\theta }_1\cos \left({\theta }_2+{\theta }_4\right)-200\sin {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\hfill \\ \hfill & +\sin {\theta }_1\cos {\theta }_2\left(1200+d_3\right)\hfill \\ \hfill n_z& =-\cos \left({\theta }_2+{\theta }_4\right)\sin {\theta }_5\hfill \\ \hfill o_z& =\sin \left({\theta }_2+{\theta }_4\right)\hfill \\ \hfill a_z& =\cos \left({\theta }_2+{\theta }_4\right)\cos {\theta }_5\hfill \\ \hfill p_z& =-\left(\cos {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\cos {\theta }_5-\sin {\theta }_1\sin {\theta }_5\right)\left(200+d_6\right)\hfill \\ \hfill & +400\cos {\theta }_1\cos \left({\theta }_2+{\theta }_4\right)-200\cos {\theta }_1\sin \left({\theta }_2+{\theta }_4\right)\hfill \\ \hfill & +\cos {\theta }_1\cos {\theta }_2\left(1200+d_3\right)\hfill \end{array}\right.$$

(3)

Therefore, based on the data detected by angle sensors and displacement sensors, Equation (3) can be used to calculate the spatial position and attitude of the drilling rig relative to the coordinate system of the manipulator base coordinate.

3. Hybrid Visual Servo System

3.1. Method for Positioning Drilling Hole

This article presents a binocular vision-based positioning system to determine the spatial position of drilling holes. The binocular cameras are mounted on the operating platform of the anchor-drilling robot. Both cameras have identical parameters. Their imaging planes are coplanar, and the optical axes pass through the center of the image planes and are perpendicular to them. The binocular vision system utilizes the Silver Bull Microelectronics R132 vision module, primarily applied in fields such as robotics, consumer electronics, drones, and 3D scanning. The visual sensor has a maximum resolution of 1280 × 800@60fps, with a baseline distance of 48 mm; the sensing depth range is from 0.25 m to 3.0 m, meeting the working space requirements in underground coal mines; the depth error at the maximum sensing depth range is approximately 2%, ensuring detection accuracy; the field of view is 88° × 58°, ensuring that the robot has a broader field of vision. The principle diagram of this system is shown in Figure 5. In the diagram, $O_W{X}_W{Y}_W{Z}_W$, $O_l{X}_l{Y}_l{Z}_l$, and $O_r{X}_r{Y}_r{Z}_r$ represent the world coordinate system, the left camera coordinate system, and the right camera coordinate system, respectively. $O_l$ and $O_r$ denote the optical centers of the left and right cameras; $C_l$ and $C_r$ represent the imaging planes of the left and right cameras; $u_l{v}_l$ and $u_r{v}_r$ denote the pixel coordinate systems of the left and right image planes; B indicates the distance between the optical centers of the left and right cameras, which is the stereoscopic camera baseline distance; and f represents the focal length of the cameras. Both the baseline distance and the camera parameters can be obtained through camera calibration.

For any arbitrary target point A in space, we assume that the coordinates of point A in the world coordinate system are $A(X,Y,Z)$, where $a_l(u_1,v_1)$ and $a_r(u_2,v_2)$ correspond to the imaging points of point A in the left and right image planes, respectively. The model described herein conforms to an ideal binocular stereoscopic measurement model. In this model, $a_l$ and $a_r$ in the pixel coordinate system have equal vertical coordinates, which are denoted as $v_1=v_2$. The difference between their horizontal coordinates, $u_1-u_2$, is a disparity value. The principle of binocular vision positioning is based on the realization of image disparity values. Based on the principles of pinhole imaging and the geometric relationships indicated in the image, the following expressions can be derived:

$$\begin{array}{l}X=\frac{u_1}{u_1-u_2}\ast B,\\ Y=\frac{v_1}{u_1-u_2}\ast B,\\ Z=\frac{f}{u_1-u_2}\ast B.\end{array}$$

(4)

The primary challenge in determining the spatial position of drilling holes using a binocular vision system is the identification of the drilling holes. Generally, target recognition and image segmentation rely on features based on shape, color, and texture. However, in coal-mine tunnels, distinguishing between the surrounding rock and steel stripes by color is challenging because of their similar hues. Similarly, the lack of distinct texture information complicates target recognition and segmentation. The drilling holes were uniformly distributed on the steel stripes and were circular with the same radius. Therefore, the identification and segmentation of anchor-drill-hole images as well as the fitting of drill holes and the calculation of their circular center pixel coordinates can be achieved using an improved Hough circle fitting method [32]. Subsequently, the spatial coordinates of the anchor drill holes were calculated using Equation (4). By transforming the coordinate systems, the spatial position of the anchor drill holes in the coordinate system of the drilling robot’s body can be ascertained.

3.2. Hybrid Visual Servo Control Strategy

3.2.1. PBVS Control System

Assuming that the center of the target drilling hole is denoted as P, we can compute its spatial coordinates in the binocular visual coordinate system by employing the binocular visual positioning method described above, which is expressed as $P_{C1}(X,Y,Z)$. $T_{C1}^B$ represents the transformation relationship between the coordinate system of the binocular camera and the coordinate system of the manipulator base. Hence, the spatial position of the center point P of the target drilling hole in the coordinate system of the manipulator base can be expressed as follows:

$$P_d=P_B=T_{C1}^B{P}_{C1}=\left(p_{dx},p_{dy},p_{dz}\right)$$

(5)

In Equation (5), $P_d$ represents the spatial coordinates of the center of the target drilling hole in the coordinate system of the manipulator base, which can also be considered as the spatial position that the drilling rig intends to reach.

At the same time, the angles or displacements of corresponding joints of the manipulator can be obtained by using the measurement data from the above-mentioned angle sensors and displacement sensors. By substituting these values into Equation (3), the transformation matrix $T_E^B$, which represents the coordinate system of the drilling rig relative to the base coordinate system of the manipulator, can be obtained.

$$T_E^B=T_1^0{T}_2^1{T}_3^2{T}_4^3{T}_5^4{T}_6^5=\left[\begin{array}{cc}{R}_E^B& P_E^B\\ 0& 1\end{array}\right]$$

(6)

In Equation (6), $T_i^{i-1}$ represents the transformational relation between the coordinate system of the i-th joint and the coordinate system of (i − 1)-th joint. Based on observation, the current position of the drilling rig relative to the coordinate system of the manipulator base can be expressed as $P_c=P_E^B=\left(p_{cx},p_{cy},p_{cz}\right)$.

Based on the initial position, the desired position for the drilling rig, and the kinematic model of the manipulator, we executed trajectory planning using a quintic polynomial. This method ensures a rapid and smooth approach to shifting the drilling rig toward the desired position. The movement of the drilling rig was halted when its proximity to the designated position was below a predetermined threshold, and the experimental results show that the threshold is set to 10 cm.

Assume that the motion function of the drilling-rig position concerning time is as follows:

$$s(t)=a_0+a_{1}t+a_2{t}^2+a_3{t}^3+a_4{t}^4+a_5{t}^5$$

(7)

where $a_i$ is the quintic polynomial coefficient. The angular velocity and angular acceleration functions of the drilling rig are expressed as follows:

$$\begin{array}{l}{s}^{\prime }\left(t\right)=a_1+2a_{2}t+3a_3{t}^2+4a_4{t}^3+5a_5{t}^4\\ s^{″}\left(t\right)=2a_2+6a_{3}t+a_2{t}^2+12a_4{t}^2+20a_5{t}^3\end{array}$$

(8)

Here, $t_0$ and $t_f$ represent the start and stop times of joint motion, respectively; $s(t_0)$ and $s(t_f)$ represent the initial position and target position of the joint, respectively. Thus, substituting the initial position and target position, angular velocity, and angular acceleration into Equation (8) yields the following:

$$\begin{array}{l}s(t_0)=a_0+a_1{t}_0+a_2{t_0}^2+a_3{t_0}^3+a_4{t_0}^4+a_5{t_0}^5\\ s^{\prime }(t_0)=a_1+2a_2{t}_0+3a_3{t_0}^2+4a_4{t_0}^3+5a_5{t_0}^4\\ s^{″}(t_0)=2a_2+6a_3{t}_0+a_2{t_0}^2+12a_4{t_0}^2+20a_5{t_0}^3\\ s(t_f)=a_0+a_1{t}_f+a_2{t_f}^2+a_3{t_f}^3+a_4{t_f}^4+a_5{t_f}^5\\ s^{\prime }(t_f)=a_1+2a_2{t}_f+3a_3{t_f}^2+4a_4{t_f}^3+5a_5{t_f}^4\\ s^{″}(t_f)=2a_2+6a_3{t}_f+a_2{t_f}^2+12a_4{t_f}^2+20a_5{t_f}^3\end{array}$$

(9)

The equations above can be written in matrix form as follows:

$$\left[\begin{array}{cccccc}1& t_0& {t_0}^2& {t_0}^3& {t_0}^4& {t_0}^5\\ 1& t_f& {t_f}^2& {t_f}^3& {t_f}^4& {t_f}^5\\ 0& 1& 2t_0& 3{t_0}^2& 4{t_0}^3& 5{t_0}^4\\ 0& 1& 2t_f& 3{t_f}^2& 4{t_f}^3& 5{t_f}^4\\ 0& 0& 2& 6t_0& 12{t_0}^2& 20{t_0}^3\\ 0& 0& 2& 6t_f& 12{t_f}^2& 20{t_f}^3\end{array}\right]\left[\begin{array}{c}{a}_0\\ a_1\\ a_2\\ a_3\\ a_4\\ a_5\end{array}\right]=\left[\begin{array}{c}s(t_0)\\ s(t_f)\\ s^{\prime }(t_0)\\ s^{\prime }(t_f)\\ s^{″}(t_0)\\ s^{″}(t_f)\end{array}\right]$$

(10)

To ensure the smooth motion of the manipulator, the speed and acceleration of the manipulator were set to zero at the start and end times:

$${[\begin{array}{cccc}{s}^{\prime }(t_0)& s^{\prime }(t_f)& s^{″}(t_0)& s^{″}(t_f)\end{array}]}^T=0$$

(11)

If Equation (10) is rewritten in the form $AB=P$, then $B=A^{-1}C$, and the coefficients of the quintic polynomial can be obtained by solving this equation.

3.2.2. IBVS Control System

Owing to the limits of the precision of long-range positioning using binocular vision, an IBVS control system was employed when the distance between the drilling rig and the drilling-hole center was less than 10 cm. This method controls the motion of the drilling rig to align it with the drilling-hole center. IBSV control utilizes visual sensors to capture image features and establishes a mapping relationship between the change rate of these image features and that of the manipulator end-effector. Consequently, the motion of the manipulator is controlled based on the change rates of these image features, thereby accomplishing specific tasks. In this study, the image features used were the pixel coordinates of the four vertices of the bounding rectangle of the drilled hole. The monocular camera used in the IBVS system is the MV-EM120M industrial camera manufactured by Microvision Technology (Xi’an, China), installed at the end of the drilling rig with its optical axis parallel to the drill rig axis. This camera features high resolution, high accuracy, and low noise, making it widely utilized in areas such as component inspection, intelligent transportation, and life science imaging. The MV-EM120M industrial camera has a maximum resolution of 1280 × 960, a pixel size of 3.75 μm × 3.75 μm, an effective photosensitive area of 4.8 mm × 3.6 mm, and a maximum frame rate of 40 fps. It is capable of capturing images of target anchoring holes and performing image-based visual servo control.

Assuming that a spatial point $Q(X_Q,Y_Q,Z_Q)$ feature coordinates in the camera coordinate system and that its corresponding imaging point on the image plane is q, the relationship between these points can be established based on the principle of pinhole imaging as follows:

$$\left\{\begin{array}{l}x=\frac{X_Q}{Z_Q}=\frac{u-u_0}{f}\\ y=\frac{Y_Q}{Z_Q}=\frac{v-v_0}{f}\end{array}\right.$$

(12)

In image plane coordinates, $u_0$ and $v_0$ denote the pixel coordinates of the principal point, whereas $(u,v)$ represents the pixel coordinate of point Q on the imaging plane. Differentiating both sides of (12) simultaneously yields the following:

$$\left\{\begin{array}{l}\dot{x}=\frac{\dot{X}}{Z}-\frac{X\dot{Z}}{Z^2}=\frac{\dot{X}-x\dot{Z}}{Z}\\ \dot{y}=\frac{\dot{Y}}{Z}-\frac{Y\dot{Z}}{Z^2}=\frac{\dot{Y}-y\dot{Z}}{Z}\end{array}\right.$$

(13)

Assuming that the linear and angular velocities of the camera are denoted as $v_c={(v_x,v_y,v_z)}^T$ and ${\omega }_c={({\omega }_x,{\omega }_y,{\omega }_z)}^T$, respectively, within the camera coordinate system, the kinematic equation describing the time-varying position of point Q can be represented as $\dot{X}=-v_x-{\omega }_c\times X$:

$$\left\{\begin{array}{l}\dot{X}=-v_x-{\omega }_{y}Z+{\omega }_{z}Y\\ \dot{Y}=-v_y-{\omega }_{z}X+{\omega }_{x}Z\\ \dot{Z}=-v_z-{\omega }_{x}Y+{\omega }_{y}X\end{array}\right.$$

(14)

By substituting Equation (14) into Equation (13), we can derive the mapping relationship between the motion velocity of point Q in the image coordinate system and its motion velocity in the camera coordinate system, which is expressed as follows:

$$\left[\begin{array}{l}\dot{x}\\ \dot{y}\end{array}\right]=\left[\begin{array}{cccccc}-\frac{1}{Z}& 0& \frac{x}{Z}& xy& -\left(1+x^2\right)& y\\ 0& -\frac{1}{Z}& \frac{y}{Z}& 1+y^2& -xy& -x\end{array}\right]\left[\begin{array}{c}{v}_x\\ v_y\\ v_z\\ {\omega }_x\\ {\omega }_y\\ {\omega }_z\end{array}\right]$$

(15)

In an IBVS control system, the image feature error is defined as given:

$$e_2=S-S^{\ast }$$

(16)

Here, $S$ and $S^{\ast }$ represents the current image feature value and the desired feature value. In this study, we selected the four vertices of the circumscribed square of the drilling hole as feature points. The coordinates of these points in the image coordinate system constitute the image feature $S$. Thus, Equation (15) can be expressed as follows:

$$\left[\begin{array}{c}{\dot{x}}_1\\ {\dot{y}}_1\\ {\dot{x}}_2\\ {\dot{y}}_2\\ {\dot{x}}_3\\ {\dot{y}}_3\\ {\dot{x}}_4\\ {\dot{y}}_4\end{array}\right]=\left[\begin{array}{cccccc}-1/Z_1& 0& x_1/Z_1& x_1{y}_1& -\left(1+{x_1}^2\right)& y_1\\ 0& -1/Z_1& y_1/Z_1& 1+{y_1}^2& -x_{1}y& -x_1\\ -1/Z_2& 0& x_2/Z_2& x_2{y}_2& -\left(1+{x_2}^2\right)& y_2\\ 0& -1/Z_2& y_2/Z_2& 1+{y_2}^2& -x_2{y}_2& -x_2\\ -1/Z_3& 0& x_3/Z_3& x_3{y}_3& -\left(1+{x_3}^2\right)& y_3\\ 0& -1/Z_3& y_3/Z_3& 1+{y_3}^2& -x_3{y}_3& -x_3\\ -1/Z_4& 0& x_4/Z_4& x_4{y}_4& -\left(1+{x_4}^2\right)& y_4\\ 0& -1/Z_4& y_4/Z_4& 1+{y_4}^2& -x_4{y}_4& -x_4\end{array}\right]\left[\begin{array}{c}{v}_x\\ v_y\\ v_z\\ {\omega }_x\\ {\omega }_y\\ {\omega }_z\end{array}\right]$$

(17)

Equation (17) can be rewritten in the form ${\dot{e}}_2=L_S{v}_c$, where ${\dot{e}}_2$ denotes the velocity of the feature points, $L_S$ denotes the Jacobian matrix of the image, and $v_c$ represents the velocity of the end-effector. In practical manipulator control systems, the controller output is the joint angular velocity. Therefore, one must compute the mapping relationship between the velocity of the manipulator’s end-effector and the joint velocities of the manipulator, i.e., the manipulator’s Jacobian matrix $J_r$.

$$v_c=J_r\dot{\theta }$$

(18)

Substituting Equation (18) into Equation (17) yields the following:

$${\dot{e}}_2=L_S{v}_c=L_S{J}_r\dot{\theta }$$

(19)

The matrix $J=L_S{J}_r$ is designated as the composite Jacobian matrix, which encapsulates the mapping relationship between the angular velocities of the manipulator’s joints and the velocity of the change in the image feature values. To ensure an exponential and decoupled decrease in the image feature error, we set ${\dot{e}}_2=-\lambda e_2$. Thus, the expression above can be reformulated as follows:

$$v_c=-\lambda L_S^{-1}{e}_2$$

(20)

If the matrix $L_S$ is not a square matrix, then $L_S^{-1}$ does not exist. In cases where the number of rows in the matrix $L_S$ exceeds the number of columns, i.e., $\dot{S}=L_S{v}_c$, there may not be a solution. Conversely, if the number of rows in the matrix $L_S$ is less than the number of columns, then $L_S^{-1}$ has multiple solutions. The following can be established by employing the Moore–Penrose pseudoinverse for resolution:

$$J^{+}={\left(J^{T}J\right)}^{-1}{J}^T$$

(21)

Hence, the control law for the actual system is represented by $\dot{\theta }=-\lambda J^{+}{e}_2$.

3.2.3. Jacobian Matrix Estimation

In the research of IBVS, the efficient and precise computation of the Jacobian matrix occupies a central position. As depicted in Equation (17), we observe that the formation of the image Jacobian matrix involves the coordinates of feature points as well as their depth information. However, in the application of monocular vision systems, the depth information of feature points is often challenging to estimate directly and accurately, posing a significant challenge in solving the image Jacobian matrix. Therefore, exploring effective methods to indirectly estimate or approximate this depth information and estimate the Jacobian matrix becomes crucial for enhancing the performance of IBVS control systems.

PSO is an optimization method based on swarm intelligence, whose fundamental principle involves simulating the social behavior of bird flocks, wherein collective information sharing enables the swarm to find the optimal destination. This collective behavior is harnessed to seek the optimal solution to a given problem.

Assume that the PSO algorithm conducts search and optimization in a D-dimensional space, where a population of n particles forms the swarm $X=(X_1,X_2,\dots X_n)$, with each particle i represented as a D-dimensional vector $X_i={(x_{i1},x_{i2},\dots x_{iD})}^T$, denoting the position attributes of the i-th particle and corresponding to potential solutions within the problem domain. Subsequently, after determining the position $X_i$ of each particle, their fitness values are computed using an appropriate fitness function. The velocity of the i-th particle is denoted as $V_i={(V_{i1},V_{i2},\dots V_{iD})}^T$, its individual best position represented by $P_i={(P_{i1},P_{i2},\dots P_{iD})}^T$, and the global best position of the entire swarm denoted as $P_g={(x_{g1},x_{g2},\dots x_{gD})}^T$.

During each iteration, particles update their velocity and position based on individual and swarm best values. The velocity update formula can be expressed as follows:

$${V_{id}}^{k+1}=w{V_{id}}^k+c_1{r}_1\left({P_{id}}^k-{X_{id}}^k\right)+c_2{r}_2\left({P_{gd}}^k-{X_{id}}^k\right)$$

(22)

In Equation (22), ω represents the inertia weight; $d=1,2,\dots D$ denotes the dimension index of particles; $i=1,2,\dots n$ represents the particle index; k denotes the current iteration count; $c_1$ and $c_2$ represent the individual learning factor and the swarm learning factor respectively, which are typically non-negative constants; $r_1$ and $r_2$ are random numbers between [0, 1] used to introduce randomness into the search process; $V_{id}^k$ denotes the velocity vector of particle i in the d-th dimension at the k-th iteration; $X_{id}^k$ represents the position vector of particle i in the d-th dimension at the k-th iteration; $P_{id}^k$ represents the historical best position of particle i in the d-th dimension up to the k-th iteration, i.e., the best solution (individual) obtained by the i-th particle after the k-th iteration. $P_{gd}^k$ denotes the historical best position of the particle swarm in the d-th dimension up to the k-th iteration, i.e., the best solution obtained by the particle swarm after the k-th iteration.

Therefore, by utilizing the Jacobian matrix from the previous time step, the incremental joint angles of the manipulator, and the current image feature error values, it is possible to construct an objective function concerning the current time-step Jacobian matrix. This function facilitates the transformation of the iterative estimation of the image Jacobian matrix into an optimization problem aiming to minimize the objective function.

Equation (19) depicts the relationship between the changing speed of the feature point and the changing speed of the joint variable, where $J$ represents the composite Jacobian matrix. Discretizing Equation (19) yields the following:

$$e_k-e_{k-1}=J_k·\left(q_k-q_{k-1}\right)$$

(23)

When the composite Jacobian matrix at time step k is denoted as $J_k$, and the error of feature points is denoted as $\Delta e_k=e_k-e_{k-1}$, the incremental joint angles of the manipulator can be expressed:

$$\Delta q_k=q_k-q_{k-1}=J_k^{+}·e_k$$

(24)

Here, $J_k^{+}$ represents the pseudo-inverse of the composite Jacobian matrix $J_k$. By continuously updating and iterating the joint angles of the manipulator using Equation (24), motion control of the manipulator can be achieved, thereby aligning the drilling rig with the center of the target drilling hole.

From Equation (19), the relationship between the changing speed of the feature point and the changing speed of the joint variable at time step k + 1 can be obtained as follows:

$${\dot{e}}_{k+1}=J_{k+1}·{\dot{q}}_{k+1}$$

(25)

Discretizing Equation (25) yields the following:

$$e_{k+1}-e_k=J_{k+1}·\left(q_{k+1}-q_k\right)$$

(26)

$$\Delta e_{k+1}=J_{k+1}·\Delta q_{k+1}$$

(27)

The composite Jacobian matrix can be expressed as follows:

$$J_{k+1}={\left[\begin{array}{ccc}{x}_{11}& \cdots & x_{1n}\\ ⋮& \ddots & ⋮\\ x_{m1}& \cdots & x_{mn}\end{array}\right]}_{m\times n}$$

(28)

In Equation (28), m and n represent the degree of freedom of the manipulator and the number of feature point coordinates, respectively. As a result, the Equation (27) can be written in the form of the following equations:

$$\left\{\begin{array}{c}\Delta e_1=x_{11}·\Delta q_1+\cdots +x_{1n}·\Delta q_n\hfill \\ \hfill ⋮\hfill \\ \Delta e_m=x_{m1}·\Delta q_1+\cdots +x_{mn}·\Delta q_n\hfill \end{array}\right.$$

(29)

In Equation (29), $\Delta e_i$ and $\Delta q_i$, respectively, denote the i-th elements of the vectors $\Delta e_{k+1}$ and $\Delta q_{k+1}$. Since the angle increment $\Delta q_{k+1}$ at time step k + 1 can only be obtained after solving $J_{k+1}$, the joint angle increment at the previous time step $\Delta q_{k+1}=\Delta q_k=q_k-q_{k-1}$ is used instead of the current joint angle increment. Based on the aforementioned, the objective functions can be formulated for minimization:

$$\begin{array}{c}\min y_1=x_{11}·\Delta q_1+\cdots +x_{1n}·\Delta q_n-\Delta e_1\\ \min y_2=x_{21}·\Delta q_1+\cdots +x_{2n}·\Delta q_n-\Delta e_2\\ \hfill ⋮\hfill \\ \min y_1=x_{m1}·\Delta q_1+\cdots +x_{mn}·\Delta q_n-\Delta e_m\end{array}$$

(30)

During the iterative estimation process of the Jacobian matrix, when the sampling period is small, the change between adjacent Jacobian matrices also decreases accordingly. Therefore, it is possible to utilize the information from the previous time step’s Jacobian matrix to assist in estimating the current time step’s Jacobian matrix. This is achieved by defining an objective function to minimize the Frobenius norm of the difference between the current and previous time step’s Jacobian matrices:

$$\min F=\alpha \left(y_1^2+y_2^2+\cdots +y_m^2\right)+\beta ·y_{m+1}^2$$

(31)

In the Equation (31), α and β represent weighting coefficients. Increasing α enhances the sensitivity of the function to errors, while increasing β directs the function to focus more on minimizing the change between adjacent Jacobian matrices. Therefore, solving for a set of independent variables that satisfy Equation (31) achieves an approximate estimation of the current time step’s Jacobian matrix.

In the context of estimating the Jacobian matrix using the PSO algorithm, the elements of the image Jacobian matrix $J_{k+1}$ in Equation (28) are defined as the positions of particles. Equation (31) serves as the fitness function in the iterative process, and the PSO algorithm is employed to optimize the particles. Hence, the optimization process is essentially an approximation of the Jacobian matrix.

4. Simulation and Experiment

4.1. Accuracy of Visual Positioning Method

To evaluate the accuracy of the visual positioning method proposed, we established an experimental platform in a laboratory environment. As depicted in Figure 6, the experimental platform primarily consists of an anchor-drilling robot, a model of a roof steel stripe, a binocular vision system, and a digital total station. There are six drilling holes in the steel stripe, which are the same size and evenly spaced. The binocular camera is located on the platform below the temporary support device of the anchor-drilling robot, tasked with capturing images of the roof steel stripe. The digital total station serves as a high-precision measurement tool capable of simultaneously assessing the spatial position and orientation information of target points. Employing the digital total station, the spatial position of the drilling-hole center is measured in the reference coordinate system, which is then regarded as the true value of the drilling hole’s spatial position. The experimental process focuses on the mechanical manipulator on the right side as the subject of study.

To verify the applicability of the drilling-hole visual positioning method described in this paper, the experimental process involved randomly adjusting the position of the steel stripe model and repeating the experiment three times. For the same drilling hole at the same location, the spatial coordinates of its center were measured using the method described in this paper, the existing method, and a digital total station. The spatial position of the drilling-hole center obtained via digital total station measurement was regarded as the true value. The existing method primarily relies on manual determination of the drilling-hole center position, moving the drilling rig to the drilling-hole center, and thus, the spatial position of the drill rig end is regarded as the manually determined drilling-hole center position. The results of the positioning experiments for the drilling-hole centers are shown in

Table 2. Experimental results of visual positioning for the center of the drilling hole.

		Position 1			Position 2			Position 3
		Proposed Method (mm)	Existing Method (mm)	Total Station (mm)	Proposed Method (mm)	Existing Method (mm)	Total Station (mm)	Proposed Method (mm)	Existing Method (mm)	Total Station (mm)
drilling hole1	X	−862.5	−859.7	−868.8	−814.8	−807.1	−809.8	−832.5	−841.9	−837.8
	Y	457.3	453.8	463.1	422.6	412.6	419.7	456.7	468.9	462.7
	Z	1214.8	1216.6	1209.6	1201.8	1206.8	1204.6	1204.8	1206.2	1198.6
drilling hole2	X	−558.6	−568.7	−564.3	−515.6	−505.6	−512.3	−531.7	−527.6	−538.3
	Y	449.8	459.8	456.7	425.6	430.6	423.1	451.8	464.3	455.9
	Z	1211.4	1201.4	1204.8	1207.4	1211.4	1206.8	1207.4	1200.9	1204.8
drilling hole3	X	−257.7	−267.1	−261.7	−217.7	−213.9	−211.7	−229.7	−239.1	−236.6
	Y	442.6	456.2	449.3	421.4	435.9	426.3	448.2	456.8	449.3
	Z	1209.4	1208.4	1203.4	1211.3	1201.3	1206.4	1202.4	1208.6	1203.4
drilling hole4	X	43.9	41.9	37.4	83.7	79.8	88.1	68.8	58.8	62.9
	Y	445.8	446.8	451.4	424.8	419.8	428.4	445.7	461.7	451.4
	Z	1211.8	1201.8	1205.2	1206.3	1203.3	1208.7	1208.8	1211.3	1203.2
drilling hole5	X	344.8	334.8	338.8	384.8	397.2	390.2	371.9	369.9	366.4
	Y	439.7	453.2	446.3	431.7	419.2	426.7	439.8	457.2	446.7
	Z	1206.4	1196.4	1203.4	1207.4	1213.8	1211.8	1203.2	1206.8	1204.7
drilling hole6	X	644.3	634.6	638.1	690.3	684.6	688.1	675.8	664.2	669.1
	Y	432.8	447.6	439.5	428.0	423.8	433.5	434.1	449.8	439.9
	Z	1205.6	1197.6	1204.9	1211.2	1209.6	1204.9	1209.3	1208.7	1211.6

. This table records the positioning results for six drilling holes on a steel stripe at three different locations, determined by the three measurement methods.

To assess the accuracy of the positioning methods described in this study, the spatial coordinates of drilling-hole centers obtained by the proposed method and the existing method are compared with those obtained results using a digital total station. Additionally, distances between the spatial positions of drilling-hole centers obtained by these two methods and those obtained using the total station were statistically analyzed, as shown in Figure 7. Figure 7a–c depict the data of the drilling-hole center positioning results on the XYZ axes, while Figure 7d illustrates the distances between drilling-hole center spatial positions obtained using different measurement methods.

In Figure 7a–c, the orange, green, and purple lines, respectively, represent the spatial positions of the drilling-hole centers along the coordinate axes obtained by the method proposed in this paper, the existing method, and the digital total station method. It can be observed that the three curves in Figure 7a–c exhibit similar trends. The differences between the results of the proposed method and the digital total station measurements along the XYZ axes fluctuate within the ranges of (−6.0 mm, 6.9 mm), (−6.9 mm, 5.0 mm), and (−4.4 mm, 6.6 mm), respectively, while the differences between the existing method and the digital total station measurements fluctuate within the ranges of (−8.3 mm, 10.7 mm), (−9.7 mm, 10.5 mm), and (−7.3 mm, 8.1 mm), respectively. Overall, the distance between the orange and purple lines is closer, indicating that the results of the proposed method are closer to the true values.

In Figure 7d, the orange and green lines, respectively, represent the distances between the spatial positions of the drilling holes determined by the proposed method and the existing method, and those obtained by the total station measurements. It can be seen that the overall values of the orange curve are slightly smaller than those of the green curve. The range of variation for the orange curve is (4.2 mm, 11.1 mm) with an average value of 8.7 mm, while the range for the green curve is (6.4 mm, 14.8 mm) with an average value of 10.9 mm. This indicates that the distance between the measurement results of the proposed method and the total station measurements is smaller than that between the existing method and the total station measurements, demonstrating that the proposed method yields more accurate positioning results.

4.2. Experimental of Hybrid Visual Servoing Control Method

To validate the effectiveness of the proposed hybrid visual servo control method in achieving rapid and precise alignment between the drilling rig and the drilling hole, this study conducted comprehensive experimental verification using the experimental platform described previously. In the experiments, we set the initial position coordinates of the drilling rig relative to the base coordinate system of the manipulator as (250 mm, −433 mm, 866 mm), while the position coordinates of the center of the target drilling hole relative to the base coordinate system of the manipulator were (600 mm, −1039 mm, 0 mm). Throughout the experiment, we continuously recorded the spatial coordinates of the drilling rig relative to the base coordinate system of the manipulator, and the results are presented in Figure 8.

As we can see, the blue, orange, and purple curves, respectively, depict the variations of the drilling rig’s position relative to the base coordinate system of the manipulator along the XYZ axes. In the first stage, the experiment employed a PBVS control method to guide the motion of the drilling rig, demonstrating good stability during this stage. When t = 470 ms, the spatial coordinates of the drilling rig reached (588.89 mm, −1021.81 mm, 97.87 mm), at which point the distance between the drilling rig and the target drilling-hole center shortened to 99.81 mm, below the preset threshold of 100 mm. Under these conditions, the visual servo control method smoothly transitioned from PBVS to IBVS.

In the second stage, using the coordinates of the four vertices of the bounding rectangle of the target drilling hole as feature points, we constructed an IBVS control method based on the method described in this paper to continue guiding the motion of the drilling rig. As shown in Figure 8, the spatial coordinates of the drilling rig relative to the base coordinate system of the manipulator gradually approached the coordinates of the target drilling-hole center. When t = 930 ms, the spatial coordinates of the drilling rig reached (599.31 mm, −1037.84 mm, 6.57 mm), at which point the distance between the drilling rig and the target drilling-hole center was only 6.71 mm. This result meets the requirements of the tunnel roof support specifications, successfully achieving alignment between the drilling rig and the target drilling-hole center, thereby providing an accurate positioning foundation for subsequent drilling and anchoring tasks.

To validate the effectiveness of the hybrid visual servo control method proposed in this paper for improving the efficiency of rapid and precise alignment between the drilling rig and the anchor-drilling-hole center, we employed two control methods: the method proposed in this paper and manual operation. In the manual operation method, workers with extensive experience were selected to operate the anchor-drilling robot. The experiment was repeated five times, and the duration of both control methods was recorded in detail. Spatial position data of the drilling rig in the base coordinate system of the manipulator were measured using a digital total station. The time consumption and the position error between the proposed method and the manual operation method are shown in Figure 9.

During the five experiments, when the drilling rig reached the target drilling-hole center under the control of the method proposed in this paper, the errors in the XYZ axes fluctuated within the ranges of (−3.8 mm, 4.2 mm), (−7.2 mm, 6.5 mm), and (−12.3 mm, 9.8 mm), with average errors of 0.6 mm, 0.9 mm, and 1.5 mm, respectively. In contrast, the errors in the XYZ axes under manual operation fluctuated within wider ranges of (−10.6 mm, 7.9 mm), (−12.8 mm, 14.1 mm), and (−9.4 mm, 12.4 mm), with corresponding average errors of 0.8 mm, 1.9 mm, and 2.5 mm, respectively. The errors in each direction under the method proposed in this paper are smaller than those under manual operation, with an overall average error reduction of approximately 41.2%. Especially for workers with average operational skills, the error advantage of the method proposed in this paper will be more significant.

In terms of time consumption, the average times of the proposed method and the manual operation method were 10.2 s and 14.5 s, respectively. The time consumption refers to the time taken for the entire process of moving the end-effector of the manipulator from its current position to the center of the drilling hole (desired position), which consists of two stages: The first stage involves controlling the end-effector to approach the desired position, while the second stage aims to precisely align the end-effector with the target drilling-hole center. In the first stage of approaching the drilling-hole center, the time of the two methods were similar. However, in the second stage of achieving precise alignment, due to the limitations of poor visibility and differences in workers’ operational skills, the manual operation required repeated adjustments, resulting in a significant increase in time consumption.

In summary, compared to manual operation, the hybrid visual servo control method proposed in this paper not only demonstrates significant advantages in error control but also significantly outperforms manual operation in terms of operation time. This fully validates the effectiveness of the method in improving the efficiency of rapid and precise alignment between the drilling rig and the drilling hole.

5. Conclusions

To rapidly and accurately align a drilling rig with drilling holes during coal-mine roof support operations, this study introduces a robotic teleoperation system alongside a hybrid visual servo control strategy. A visual positioning system was established by employing triangulation principles to calculate the spatial location of drilling holes and determine the target position of the drilling rig. To guarantee both efficiency and precision, we proposed a hybrid control strategy by combining the PBVS and IBVS control, where the former facilitates an effective approach to the target drilling hole, and the latter ensures high-precision alignment with the center of the drilling hole. Both the simulation and experimental results indicated that the hybrid visual servo control method effectively controlled the drilling rig to reach the drilling-hole center accurately and promptly. Compared with the existing manual control method, the average error is reduced by 41.2%, and the average time consumption is reduced by 4.3 s. The method described herein contributes significantly to enhancing the automation and intelligence levels in coal-mine production.

The method described in this article is crucial for aligning the drill rig with the center of the drilling hole quickly and accurately, not only enhancing the automation level of support equipment but also further improving tunnel support efficiency. Despite our efforts to construct an experimental platform in a laboratory setting and simulate conditions of low illumination and high dust concentration similar to those in underground coal mines, there still exists a disparity compared to real tunnel environments. Therefore, future endeavors will focus on promptly conducting industrial experiments in underground coal mines to validate the effectiveness of the method described in this article in actual mine-tunnel settings.