*Article* **A Smart Modular IoT Sensing Device for Enhancing Sensory Feedbacks in Surgical Robotics**

**Mafalda Rosa 1,2, Rongrong Liu 1,2, Giorgio Pitruzzello 1,2,3 and Giuseppe Tortora 3,\***

<sup>1</sup> BioRobotics Institute, Scuola Superiore Sant'Anna, 56127 Pisa, Italy


**\*** Correspondence: smartmedicaltheatre@abzero.it

**Abstract:** This paper proposes a device of sensing that could be integrated into the instruments of any surgical robot. Despite advances in robot-assisted laparoscopic surgery, the tools currently supplied to surgical robots have limited functions, due to the absence of sensorization. With this motivation, we present a preliminary work based on the design, development, and early stages of experimentation with smart and multifunctional devices of sensing for surgical tools. The proposed device of sensing has a proximity sensor, colorimetric sensor, and BLE connection for different surgical instruments to connect to each other. The proximity feedback allows the surgeon to know the distance of the instrument from a particular tissue, to operate in conditions of greater safety. With the colorimetric feedback, on the other hand, we intend to proceed to the identification of specific tissue areas with characteristics that are not typical of the physiological tissue. The results show that the device is promising and can be further developed for multiple clinical needs in robotic procedures. This system can effectively increase the functionality of surgical instruments by overcoming the sensing limitations introduced by using robots in laparoscopic surgery.

**Keywords:** robot-assisted laparoscopic surgery; surgical robot; da Vinci Research Kit; IoT

**Citation:** Rosa, M.; Liu, R.; Pitruzzello, G.; Tortora, G. A Smart Modular IoT Sensing Device for Enhancing Sensory Feedbacks in Surgical Robotics. *Appl. Sci.* **2022**, *12*, 8083. https://doi.org/10.3390/ app12168083

Academic Editor: Alessandro Gasparetto

Received: 11 July 2022 Accepted: 10 August 2022 Published: 12 August 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**1. Introduction**

The past few decades have seen an exponential growth in medical technology, particularly with regard to the application of robotics to surgery. Robotic surgery, the latest evolution of minimally invasive surgery, overcoming the limits of traditional surgery, has allowed the broadening of therapeutic horizons and represents the gold standard for various clinical applications.

Robotics is the center of modern health engineering. The first robot used in the clinical setting to obtain neurosurgical biopsies was the Puma 560 robot in 1985. Since then, more and more advanced surgical robots have been developed [1,2]. In general, the use of robotic surgery increased significantly from 2012 to 2018, with an increase from 1.8% to 15.1% for all general surgery procedures. Over the same period, the use of both laparoscopic and open surgery declined. For example, the proportional use of open surgery was 42.4% in 2012, compared to 32.4% in 2018 [3]. It has also been witnessed that the use of robotic surgery has increased rapidly and spread widely in numerous procedures during the years following the adoption of this practice in hospitals. Therefore, for most surgeons, it was already considered a safe and effective approach when clinically feasible.

Current robotic platforms are designed to incorporate advanced features that allow for increased accuracy by making the execution of operator tasks easier and safer. Additionally, surgical robots have retained the ability to perform surgical operations through smaller incisions. These characteristics aim to improve the results compared to those obtainable through traditional surgical methods. The adoption and diffusion of robotic surgery shows a positive trend in some geographical areas, especially in countries with advanced

economies. This is shown by the widespread use of the da Vinci Surgical System (Intuitive Surgical Inc., Sunnyvale, CA, USA), in a multitude of application areas [4]. Currently, the da Vinci Surgical System represents the most widespread surgical system, with over 5000 models implemented worldwide, performing over 7 million surgical procedures in different anatomical areas. The da Vinci Research Kit (dVRK) research platform fits into this context, developed through a collaboration among academic institutions to address the challenges in starting research on surgical robotics. This has led to a significant boost in the development of surgical robotics research over the past decade, and has generated new opportunities for collaboration and linking of a surgical robot to other technologies.

Among the advantages introduced by robot-assisted surgery are the reduction in tissue trauma thanks to small incisions, less bleeding, and less need for transfusions, a reduction in hospital stays and post-operative pain, a reduction in recovery times, and a quicker recovery rate in carrying out daily activities and greater ease in the execution of complex surgical tasks, which entails greater safety for the patient. On the other hand, the disadvantages of robotic surgery are mainly linked to the cost of the robotic system, the instrumentation, the system maintenance, and to the fact that, to operate the robot, very high-level skills are required on the part of the surgeon and room staff, to be acquired through specific training [5].

During an open surgery, surgeons can use their hand to locate and diagnose abnormal tissue by direct palpation; instead, in laparoscopic surgery, direct palpation is not feasible, due to the limits of the incision [6,7]. For this reason, one aspect that many studies are focusing on is the lack of haptic feedback during the surgical procedure, in addition to visual feedback [8,9]. Nevertheless, surgeons using robotic technologies could benefit from other types of feedback, such as feedbacks of color, speed, and proximity, to further broaden the fields of the application of surgical robots [10,11].

In this work, an attempt is made to restore the functions that are lost in robotic laparoscopic surgery, by using the sensorization of the surgical instrumentation. The design intention is to develop an intelligent and multifunctional sensing device to improve the performance of surgical robots, interconnect instruments, and enable, in the future, the development of AI algorithms.

### **2. Materials and Methods**

#### *2.1. Device Overview*

In this section, the overall design of the proposed device of sensing is introduced. For the implementation, the following needs have been considered and analyzed:


The main purpose of the first prototype proposed in this manuscript is to carry out the sensorization of the surgical instrument used during procedures with surgical robots, to ensure a safer interaction with the organs. For the first prototyping phase, an Arduino Nano 33 BLE Sense board is used, due to its compact size and the presence of integrated sensors. Reference is made to the colorimetric and proximity sensor integrated inside the APDS9960 unit of the Arduino board considered. In particular, the proximity sensor provides feedback on the distance between the tip of the surgical instrument and the organs, to ensure safe interaction. In addition, the colorimetric feedback allows, with the implementation of a simple neural network, to identify specific tissue areas with characteristics that are not typical of physiological tissue, such as cancerous tissue structures. This kind of recognition leads the way, in the future, to the classification of healthy tissue and diseased tissue. A 3D printed mechanical support is used to mount the electronic board to the robotic instruments, consisting of two main parts held together using a magnetic anchor. The magnets are integrated into the mechanical structure of the device, and allow for a quick

alignment of the two parts, and a stable anchoring to the instrument. The dimensions of the holder are compatible with the size of conventional da Vinci EndoWrist tools; as this device of sensing is designed for robot-assisted laparoscopic surgery, it can be integrated into instruments of the Patient Side Manipulator (PSM) of the da Vinci Research Kit (dVRK) and can be teleoperated by the Master Tool Manipulator (MTM) [12] (Figure 1).

**Figure 1.** The developed sensing device mounted on the PSM of the dVRK system.

#### *2.2. Design and Implementation of the Sensing Device*

#### 2.2.1. Mechanical Design

A mechanical support is created to allow to the Arduino Nano 33 BLE Sense board to be kept in position on the dVRK instrumentation during the execution of the validation tasks. It is conceived in two different versions to adapt to the different positioning along the instruments of the dVRK. The first version of the prototype, shown in Figure 2, allows the lying positioning on the robot instrument. This arrangement is ideal during BLE communication between the various robot arms, as there is no need to keep the sensor integrated on the board in a specific position. Furthermore, the first position is characterized by a reduced size and a better flexibility during the movements.

**Figure 2.** The CAD design of the first version of the mechanical support. (**a**) The housing for the electronic board. (**b**) The connection part for the positioning of the device along the EndoWrist tools.

The second version, shown in Figure 3, is designed for cases in which it is necessary to hold the embedded sensors that look towards the work surface. This is useful in the experimental validation phases during the detection of the color and proximity of tissues placed on the work surface.

This support, for both versions, is made up of two main parts. The first part represents the housing for the electronic board, while the second part connects to the first for the positioning of the device along the EndoWrist tools, as shown in Figures 2 and 3. The two structural portions of the mechanical support are held together using four K&J MAGNET-ICS neodymium magnets (NdFeB), for each side. The magnetic blocks have dimensions of (0.64 × 0.32 × 0.08) mm and grade N42. The magnetic anchoring enables the quick alignment of the two mechanical parts and the perfect integration of the sensing device

with the dVRK instruments. At the same time, this magnetic mechanism makes it possible to separate the two structural portions of the support after performing the tasks using the tip of the surgical instrument itself [13,14]. This operation not only saves time, but can be performed without the assistance of an assistant surgeon.

**Figure 3.** The CAD design of the second version of the mechanical support. (**a**) The housing for the electronic board. (**b**) The connection part for the positioning of the device along the EndoWrist tools.

#### 2.2.2. Hardware Components

In this research, we employ the Arduino Nano 33 BLE Sense board composed of embedded sensors to detect color, proximity, motion, temperature, humidity, audio, and more. The presence of the embedded sensors allows the board to manage numerous IoT and AI applications without requiring the presence of external sensors. This board is built upon the nRF52840 microcontroller and runs on Arm® Mbed™ OS. The processor has other important features such as Bluetooth® pairing via NFC and ultra-low mode energy consumption. For the sensorization of the device, reference made is to the APDS-9960 unit built into the Arduino Nano33 BLE Sense board, which features advanced gesture sensing, proximity sensing, digital ambient light sensing (ALS), and color sensing (RGBC). This modular unit has dimensions 3.94 × 2.36 × 1.35 mm, and incorporates an IR LED and a factory-calibrated LED driver.

The proximity detection function provides the measurement of distance via the photodiode sensing of reflected IR energy from built-in LEDs. Detect/release events are interrupt-driven and occur whenever the proximity result crosses the upper and/or lower threshold settings. The IR LED intensity is factory trimmed to eliminate the need for end-equipment calibration due to component variations. The proximity results are further improved by automatic ambient light subtraction. The proximity results are affected by three basic factors: IR LED emission, IR reception, and environmental factors, including distance to the target and the surface reflectivity. The photodiode signal is combined, amplified, and offset adjusted to optimize performance. The colour and ALS detection feature provides red, green, blue, and clear light intensity data. Each of the R, G, B, C channels has a UV and IR blocking filter and a dedicated data converter producing 16-bit data simultaneously. This architecture allows applications to accurately measure ambient light and sense colour, which enables devices to calculate colour temperature and control display backlight.

#### 2.2.3. IoT and Bluetooth Low Energy Connection

In recent years, we have seen a significant advance in digital technologies, which contributes to the current concept of the Internet of Things (IoT). At the basis of the IoT, there are "intelligent" objects that are interconnected to the exchange information owned, collected, and/or processed. The smart object must first be identifiable; that is, with a unique identifier in the digital world, and then it must be connected to transmit and receive information. These are smart connected devices that process and share all kinds of data with each other, and that can be controlled via the Internet. Into this context fit energy-efficient short-range wireless communication technologies such as Bluetooth Low Energy (BLE) [15]. This section shows how is possible to exchange information between two Arduino Nano 33 BLE Sense boards. With this communication, the various arms of the da Vinci surgical robot can be interconnected.

When a Bluetooth® connection is established, the central device will scan the surrounding devices and "listen" for the devices that transmit information, and at the same time, the device will advertise or transmit its data or information to any nearby device. As soon as the central device collects information from the peripheral device, an attempt is made to connect the peripheral device. Once the connection is established, the central device will interact with the available information to the peripheral device. This information exchange takes place using so-called services. By grouping the various device capabilities into services, central devices allow peripheral devices to quickly find, select, and interact with the desired services. Any service has a unique identifier called UUID. This code can be 16 or 32 bits long, for services with Bluetooth® specifications. One of the two Arduino Nano BLEs is configured as a central device, while the other as a peripheral device. The information shared between the two boards comes from the proximity sensor of the integrated APDS-9960 unit of the Arduino Nano 33 BLE Sense board.

For this purpose, the ArduinoBLElibrary library was used, and a service called proximityService was created with a feature called proximity\_type, as shown in Figure 4. The central device tries to establish a connection with the peripheral device, and tries to discover the service and the feature that we have specified when implementing the code. If the connection is made successfully, the Nano 33 BLE Sense board's built-in proximity sensor is activated. When a proximity value is detected by the sensor, the central device gives us feedback, through the serial monitor, on the type of distance detected (FAR, MIDDLE, or CLOSE). The value is written to the proximity\_type feature of the proximity service in the peripheral device. In addition, the on-board LED in the peripheral device lights up according to the detected value. If a distance is detected that exceeds the threshold set as FAR, the green LED will turn on; in the case of MIDDLE distance, the blue LED will be on; finally, if the object is close to the sensor, the red LED lights up.

**Figure 4.** BLE communication between two Arduino Nano 33 BLE Sense units.
