An Interventional Surgical Robot Based on Multi-Data Detection

Abstract

Vascular interventional surgery is the most common method for the treatment of cardiovascular diseases. Interventional surgical robot has attracted extensive attention because of its precise control and remote operation. However, conventional force sensors in surgical robots can only detect the axial thrust pressure of the catheter. Inspired by the function of insect antennae, we designed a structure with a thin-film force sensing device in the catheter head. Combined with the pressure sensor in the catheter clamping device, multiple sensor data were fused to predict and classify the current vascular environment using the LSTM network with 94.2% accuracy. During robotic surgery, real-time feedback of current pressure information and vascular curvature information can enhance doctors’ judgment of surgical status and improve surgical safety.

1. Introduction

With an aging population, incidence of cardiovascular disease is on the rise worldwide and has become one of the biggest diseases that harm the elderly. At present, interventional surgery has gradually become the best treatment for cardiovascular diseases due to its low trauma and fast recovery [1]. Traditional interventional surgery requires very experienced surgeons to perform it. The surgeon creates an entrance in the patient’s femoral or radial artery and sends the guide wire and catheter into the patient’s artery. They must push the guide wire and catheter along the artery to the lesion site and then perform drug delivery or stent construction [2]. Due to the complexity of the vessels and the inability of the surgeon to monitor their status in real time, a digital subtraction angiography (DSA) system is required to locate the catheter. In traditional surgery, the DSA system produces so much radiation that doctors wear very heavy lead suits to protect themselves, which makes them very tired. In the long run, surgeons often suffer from chronic diseases such as lumbar muscle strain, which seriously harm their health. In addition, long and intense procedures can cause the surgeon’s hand to tremble, which can cause the catheter to pass through blood vessels and harm the patient. Therefore, researchers in recent years have focused on robotic systems for interventional surgery, which allow surgeons to perform surgery remotely to protect themselves from radiation, while improving the stability and accuracy of surgery [3].

In recent years, several companies in the United States have developed surgical robotic systems capable of commercial application, and they have been able to be applied to a number of simple clinical surgical applications [4,5,6,7]. In addition to commercial companies, many university research groups are also working on surgical robots. The research focuses on three aspects of robot control, force detection and force feedback. Some researchers have used pressure sensors to detect the resistance encountered when advancing from the end [8,9]. Some researchers have used micro-optical fiber sensors to detect the triaxial deformation relationship between catheter head and vascular wall [10,11]. In the context of force feedback, some researchers use magnetorheological fluid as a medium to control magnetic field intensity, so as to achieve force feedback at the main end [12,13,14,15]. Some researchers used the Phantom desktop haptic device as the main controller, with integrated force feedback function [16]. In terms of control, some researchers also pay attention to robot control algorithms and improve the overall control accuracy of robots through new control algorithms [17,18].

At present, most interventional surgical robot systems adopt master-slave structure. The robot is provided with a guide wire and a catheter clamping device at the slave side. The surgeon remotely controls the controller on the master side. The slave side receives the control signal of the master side and replicates the operation from the master side, thus realizing the remote operation of the robot. As a result, surgeons do not need to wear lead suits to operate, which eases the physical burden on surgeons and protects them from radiation. At the same time, the driving motor at the secondary end can also provide high precision and stability control, avoiding error caused by the shaking of the surgeon’s hand in traditional surgery. Compared with traditional manual surgery, robotic surgery has the advantages of high precision and non-exposure to radiation. However, how to ensure the safety of robotic surgery is of upmost importance.

Collision force detection during the slave side catheter propulsion in robotic surgery is a hot research area. As the catheter moves through the blood vessel, friction can occur between the catheter and the vessel wall. When passing through the bend of the blood vessel, the catheter head is easy to collide with the wall of the blood vessel, and a large collision may even scratch the blood vessel, causing harm to the patient. Because the catheter is propelled by a motor from the end, the surgeon cannot directly feel the resistance of the catheter. Therefore, the detection of the collision force of the catheter is very important in robotic surgery. The detected force information from the slave side can be fed back to the master side, and more surgical information can be given to the operator through force feedback lever or data display feedback, which can improve the safety of the operation. In current surgical robot research, the most commonly used method is to install a pressure sensor in the catheter clamping module to detect the collision force of the catheter head through the conduction of the mechanical structure. However, due to the gap between the structures, the sensor reading will not be particularly accurate, and only the axial force can be detected, which cannot determine the deformation of the catheter head. The surgeon may misjudge the patient’s blood vessels by not having enough sensing information, pushing the catheter forward and damaging the patient’s blood vessels. Currently, catheters equipped with proximal audio sensors or micro-optical fiber sensors [10] can detect the force and deformation direction of catheter head force. Through multi-dimensional sensor data feedback, surgeons can strengthen their grasp of surgical conditions, so as to improve the safety of robotic surgery. However, the cost of this method is too high in practical application, including the cost of active components, cables and other consumables.

Inspired by insect antennae, a mounting bracket for the catheter head is designed in this paper so that two half-bridge strain gauges can be attached to the catheter head. The strain gauge data and the pressure sensor data in the clamping device were combined to carry out the tube pushing experiment of different angles of the catheter through the blood vessel and collect the three-dimensional sensing data. LSTM network is used to predict and classify the collected data, so as to realize the function of recognizing the current angle through blood vessels. Compared with traditional force detection devices, this method can detect multi-dimensional sensing information and judge the current bending state through blood vessels. Giving the operator more data feedback greatly improves the safety of surgical robots.

The remainder of this paper is structured as follows: In Section 2, the system structure of the interventional surgical robot and the working principle of the sensors are introduced. In Section 3, the experiment principle and procedure are given. In Section 4, network training and data analysis are carried out. Finally, our conclusions are given in Section 5.

2. System Description

2.1. Overview of the Surgical Robot System

In traditional interventional surgery, surgeons make an incision in an artery in the patient’s arm or thigh and insert a guide wire and catheter. The surgeon usually inserts the guide wire into the patient’s blood vessel first. The guide wire serves as a guide to the catheter, so the catheter can be pushed along the path of the guide wire. The surgeon pushes and rotates the catheter to move it along the blood vessel to the lesion. In summary, the operation of the catheter can be divided into two parts: pushing and rotating. Therefore, the catheter push device of the surgical robot system needs to be designed with two degrees of freedom to achieve catheter push and rotation.

Figure 1 illustrates the workflow of the interventional surgical robot. The robot system consists of three main parts: a master manipulator, a slave manipulator and a motion controller (IMAC-LX, Delta Tau, CA, USA). The master side consists of a system control terminal, including two operations controller, and is respectively responsible for the control guide wire and tube and rotation. The master side is located outside the operating room to protect the surgeon from radiation damage. The slave is a robotic action system that replicates the surgeon’s instructions.

The structure of the slave is shown in Figure 2. The linear slide platform is provided with a guide wire clamping module and a conduit clamping module. Two DC motors (EC-22, Maxon, Sachseln, Switzerland) are installed at the end of the platform and are responsible for rotating the tracks to propel the modules. Each clamping module is also equipped with a brushless DC motor (EC-22, Maxon, Sachseln, Switzerland), which is responsible for controlling the rotation of the catheter or guide wire through a gear structure. When the surgeon operates the master manipulator, the motion information on the master side is transmitted to the motion controller using the EtherCat communication protocol. The motion controller transmits the acquired signal to the slave and drives the slave motor to push the guide wire and catheter into the patient’s blood vessel. In the meantime, the DSA system captures the image information of the catheter and guide wire during the operation, and displays the image to the monitor, enabling the surgeon to ensure the safety of the operation.

2.2. Structure of the Catheter Push Module and Force Sensor

The structure of the catheter clamping module is shown in Figure 3. The catheter is fixed on the catheter clamping component. The advance and retreat of the catheter are controlled by the DC motor at the end, and the rotation of the catheter is controlled by the DC motor inside the clamping module. The internal DC motor turns forward or reverse at a fixed low speed, so the catheter can rotate left and right through the transfer gear. A steering engine is installed inside the clamping module, which is connected to the guide wire control unit through a steel wire. When the steering engine turns to a fixed angle, the steel wire is pulled and the guide wire control unit is pressed down to realize the role of the guide wire.

Figure 3 shows the structure of the catheter clamping module. The catheter is secured to the catheter clamping assembly. The forward and backward direction of the catheter is controlled by a DC motor at the end, and the rotation of the catheter is controlled by a DC motor which is installed inside the clamping module. The internal DC motor rotates forward or backward at a constant low speed, so the catheter can be rotated left and right by means of a transfer gear. The clamping module is equipped with a steering gear which is connected to the guide wire control unit through a steel wire. When the steering gear turns to a fixed angle, the steering gear can pull the wire and press down on the guide wire control unit to realize the clamping and loosening of the guide wire.

The module is integrated with a force sensor assembly, which is integrated under and fixed to the catheter clamp assembly. The two sides of the force sensor are fixed by sliding rails. When the catheter is pushed against resistance, the force is transmitted through the mechanical structure to the force sensor below. The schematic diagram of force conduction is shown in Figure 4. Figure 5 shows the force sensor in the module.

The collision force range of the catheter in the blood vessel is small, generally within 2N, so the force sensor with small range and high accuracy is selected (ZNLBS-IIX, Zhongnuo, China). The sensor parameters are listed in

Table 1. Specific parameters of the force sensor.

Product Model	ZNLBS-IIX
Detection range	0–3 N
Accuracy	0.1% F.S
Resolution	0.1% F.S
Zero output	±1% F.S
Sensitivity	1.0 mV/V

.

3. Experiments and Data Collection

3.1. Principle and Physical Assembly

Force detection is particularly important in robotic surgery. In the process of catheter propulsion, especially when the catheter passes through the curve of the blood vessel, the catheter may have friction or collision with the blood vessel wall and produce some resistance. How to detect and judge whether the current resistance is safe is the biggest problem at present. Most robotic systems use a master-slave structure, so the surgeon cannot directly feel the resistance of pushing the catheter from the end during surgery. Conventional force detection method in robotic surgery only relies on force sensors on the clamping module. The force information detected by this method is not accurate. When surgeons are unsure of the current vascular environment, contrast agents are often injected and the DSA system is activated to capture images. If the injection of contrast agent is implemented too many times, it will affect the patient’s health and prolong the operation time. Our goal is to establish a force detection method with multidimensional data in light of the current deficiencies of robotic manpower detection in interventional surgery. By fusing the data of multiple sensors and classifying and identifying the data by means of neural network, the bending degree of the current catheter can be judged in real time, and the force information and catheter angle information can be fed back to the surgeon, which can better ensure the safety of the operation. Many insects in the world have tentacles, such as ants, mosquitoes and beetles. They all use their antennae to compensate for a lack of vision or sense of smell. They have many tiny tactile neurons in the head of their antennae that respond to touch [19]. As they move, they can tell if it is safe by constantly touching the ground in front of them with their antennae. As shown in Figure 6, butterflies can sense the changes of the external air flow by using their antennae, and judge whether other creatures are near by the changes of the air flow [20]. Inspired by the function of insect antennae, we regard catheters as insect antennae and install corresponding detection devices in the head of catheters, so that we can make certain judgments about the vascular environment.

Some research groups monitored the stress on the catheter by installing strain gauges on the catheter head [21]. However, they only made a vague qualitative judgment on the safety of surgery based on the strain gauge data and could not achieve a more accurate quantitative analysis. On the basis of pasting strain gauge data and combining the data of strain gauge and the force sensor in the traditional clamping mechanism, we used the LSTM network to predict and classify the data series, which can identify the deformation information of the current catheter, so as to give more data feedback to the operator and improve the safety of the operation.

A strain gauge is a thin-film device for measuring stress and deformation. It consists of sensitive semiconductor materials arranged in a fixed shape. When the strain gauge is bent and deformed, its resistance value changes accordingly. Through the construction of the peripheral amplifier circuit, the change of the resistance value of the strain gauge itself can be converted into the change of voltage, and the shape variable of the strain gauge can be known through the voltage. The installation diagram of strain gauge in this paper is shown in Figure 6. The surgical catheter is cylindrical, and the strain gauge needs to be attached flat, so it is difficult to attach the strain gauge directly. The design of the support to paste the strain gauge helps to increase the paste area and ensure that the strain gauge is firm. As shown in Figure 7, two sets of stents (BFH350-6AA) were installed on the catheter head in horizontal and vertical directions, respectively, and strain gauges were pasted on each stent. The two groups of strain gauges are connected in the form of half-bridge, which has better anti-interference compared with single strain gauge measurement [22].

3.2. Data Acquisition

We set the reference voltage of the amplifier circuit of the strain gauge to 1.5 V, and adjusted the amplification factor to make the output voltage range between 0–3.3 V according to the bending degree of the conduit, which is convenient for MCU when reading the voltage signal. The sampling rates of the two groups of strain gauges in the catheter head and the force sensor in the clamping module were consistent, which is convenient for unified analysis of multiple groups of data. In order to test the reliability of the strain gauge and the force sensor, the vascular model was used to carry out the catheter push experiment. Figure 8a shows the vascular model used in a push, and the data obtained is shown in Figure 8b. The blood vessel model was 3D printed with transparent resin. During this push, the catheter passed through a vessel with a curve of about 30°. The force sensor in the clamping module detected the resistance in front of it. The strain gauge output voltage at the head also experienced a rise. As the head of the catheter passed through the curved vessel, the voltage of the strain gauge fell back to the reference voltage again. It can be seen that the strain gauge at the catheter head and the force sensor at the catheter end had an obvious response to the bending of the vessel during pushing.

In the catheter push experiment, the data collected can be regarded as a sequence of three-dimensional eigenvalues. Using the silicone tube as a blood vessel model, blood vessel models with different bending angles can be made by arbitrary bending. The timing and speed of the catheter push remained consistent except for the different bending angles of the vessel models.

4. Data Training and Results

4.1. Long Short-Term Memory Network

With the improvement of computer performance, machine learning algorithms are applied in various fields more frequently. Among them, the cyclic neural network (RNN) has been widely used. RNN networks are commonly used for tasks that target sequence data, such as image captioning, machine translation and classification prediction. The RNN network consists of circulating cells that allow information to be retained for subsequent computation. However, it also has the problem of gradient explosion and gradient disappearance in the training process [23]. Long short term memory (LSTM) network based on RNN can solve these defects. LSTM network processes sequence information through a memory unit and gate mechanism, and has long-term dependence capability, which is superior to traditional RNN [24]. As shown in Figure 9, LSTM cells, in contrast to RNNS, consist of input gates, forget gates and output gates, with cell states to regulate and protect the flow of information within the cells. To achieve this, input gate ig and process input gate control the updated input values into the memory unit, while forgetting gate fg controls what information should be retained in the previous state of the unit [25]. Subsequently, the output gate og regulates what information received in the update cell state is output and dispatched to the hidden state.

In this paper, we installed three groups of sensor devices at the slave side of the robot system. The three groups of sensors are the two groups of strain gauges in the catheter head and the force sensors in the catheter clamping module. They all output analog voltage signals, so we chose ADC inside the single chip microcomputer (STM32F030) to collect voltage and save data. The sampling rate is set at 2 Hz to ensure the synchronization of the data sequences of the three groups of sensors. The output sequence of the two groups of strain gauges in the catheter was set as α and β, and the output sequence of the force sensor in the clamping module was set as γ. In this way, three sets of sequences can be combined into a three-dimensional vector A, in which the three feature vectors are

$$A=\left\{\begin{array}{c}\alpha \\ \beta \\ \gamma \end{array}\right\}$$

(1)

Figure 10 shows the structure of the LSTM predictive classification network used in this paper. Sequence group $A$ has three feature dimensions, which are fed into the LSTM layer with 100 hidden units as the input of the network. The dropout layer sets the dropout probability to 0.5, which means that each neuron has a 50% probability of being removed, so that the training of one neuron does not depend on another neuron and the synergistic interaction between features is weakened to prevent overfitting of the network. Finally, through the full connect layer and the softmax layer, the results are converted into probabilities to realize the classification of data sequences.

Figure 10 shows the structure of the LSTM predictive classification network used by the surgical robot system. Sequence group A has three feature dimensions and is fed together as input into the LSTM layer with 100 hidden units. The dropout layer has a dropout probability of 0.5, which means that each neuron has a 50% chance of being removed, which weakens the synergy between features and prevents overfitting of the network output. Finally, through the full connection layer and softmax layer, the results are converted into probabilities to realize the classification of data series.

4.2. Experiments and Training Result

As shown in Figure 11, in order to establish data sets for network training, we used silicone tubes to make vascular models with different curviness and used them to conduct catheter push experiments. During the experiment, except for changes in the angle of the vascular model, the advancing speed and advancing time of the catheter were kept consistent to ensure the stability of the data. Considering the actual vascular environment of the human body, each 15° bend of the vascular model is classified into one category. The total number of categories is 11. Therefore, the catheter bending angle of 0–150° can be detected. In order to enrich the data set, experiments under each angle group were automatically controlled by the computer 200 times, and the catheter was rotated randomly before each advance to ensure the diversity of data, avoiding the overfitting of the network caused by less data. We set the push time of each experiment at 15 s and the sampling rate at 3 Hz. Therefore, 45 lengths of sequence data could be collected in a push experiment. A total of 2200 sets of sequence data were collected to train and verify the network under 11 different vascular angles. 70% of all data is used for network training, 15% is used for verification group and another 15% is used as test set to verify network reliability. In network training, the step size was set to 0.001, BatchSize to 32 and the maximum number of iteration rounds to 100. The training gradient diagram is shown in Figure 12. The training results are shown in

Table 2. Results of the network training.

	Cross Entropy	Error
Training group	0.0605	0.0565
Validation group	0.0676	0.0648
Test group	0.0612	0.0573

.

As can be seen from the table, the network after training has a good classification performance for the multi-dimensional sensor data on the slave side of the surgical robot system. The classification accuracy of the test set reached 94.27% through the network, which could well realize the recognition and classification of multi-dimensional sensor data, feedback the bending status information of the catheter to the main end and give surgeons more feedback information to ensure the safety of robotic surgery.

5. Conclusions

This paper introduces the development history and structural characteristics of a vascular interventional surgical robot, designs a master-slave interventional surgical robot and introduces the structure and working principle of the surgical robot. The surgeon can remotely operate the master control to push the catheter through the catheter clamp device on the slave side, reducing the surgeon’s fatigue and radiation hazards, while improving surgical accuracy. In order to solve the problem that the traditional force detection structure data of surgical robot system is not accurate and single, a stent structure is designed in this paper and a strain gauge is installed on the catheter head. The two groups of strain gauges were combined with the data of the force sensor in the clamping device, and all the data were integrated into a sequence with three-dimensional characteristics. The LSTM network was used to classify the data and the identification of the bending angle of the catheter head was successfully realized. The recognition range is 0–150°, with every 15° being one category. The test group verified that the accuracy rate was 94.27%. Compared with the traditional single force sensor detection method, the multi-dimensional force sensing data can be collected by this method and the vascular environment status can be fed back to the surgeon. The increase of feedback information can greatly improve the safety of robotic surgery. Compared to the optical fiber sensor in the catheter head, the proposed method is cheaper, but still more expensive than the conventional force sensor method. Because of the existence of the cable, it is bound to have an impact on the size of the catheter. How to reduce the cost and ensure the catheter size will be a problem we need to solve in the future.

In our next study, we will conduct experiments under more vascular models from different angles to improve the number and resolution of classification. At the same time, we have only considered the ideal case of catheter push. In practice, blood clots may exist in the blood vessels of some patients, which will affect the readings of various sensors, thus affecting the quality of network classification. In addition, blood flow in the actual operation will also produce a certain resistance to the catheter push. Therefore, in the follow-up experiment plan, we plan to add the function of a water pump to simulate blood flow and add some interference to the blood vessel model, so as to make the obtained data closer to reality in order to improve the applicability of the network.