Sewer Cleaning Robot: A Visually Assisted Cleaning Robot for Sewers

Abstract

Aiming to solve the problem of clearing obstacles in narrow and complex sewers, this paper introduces a visually assisted Sewer Cleaning Robot (SCR) for cleaning sewers with diameters ranging from 280 to 780 mm. The main work is carried out as follows: (a) A mobile platform is equipped with a pressing mechanism to press against the pipe walls in different diameters. The arm uses high-load linear actuator structures, enhancing load capacity while maintaining stability. (b) A Detection–Localization–Cleaning mode is proposed for cleaning obstacles. The YOLO detection model is used to identify six types of sewer defects. Target defects are then localized using monocular vision based on edge detection within defect bounding boxes. Finally, cutting is performed according to the localized defect positions. The feasibility of SCR in cleaning operations is validated through a series of experiments conducted under simulated pipeline conditions. These experiments evaluate its mobility, visual detection, and localization capabilities, as well as its ability to clear hard obstacles. This paper provides technical reserves for replacing human labor that use vision algorithms to assist in cleaning tasks within sewers.

1. Introduction

During heavy or torrential rains, obstacles in drainage systems can lead to severe urban flooding, resulting in casualties and significant economic losses [1,2,3]. In small- to medium-sized sewers, where space and accessibility are constrained, robotic solutions have become one of the essential approaches to trenchless pipeline rehabilitation.

In recent years, researchers have made significant advancements in the development of pipeline robots, particularly for sewer defect detection [4]. These robots were equipped with advanced technologies, such as high-resolution cameras, ultrasonic sensors, and LiDAR, to identify and classify defects like cracks, root intrusions, and debris [5]. Guo et al. [6] introduced an automated defect detection system for sewer pipelines, using a change detection approach to analyze inspection videos and images. Yin et al. [7] proposed a real-time automated defect detection framework for sewers using the YOLOv3 deep learning algorithm. Fang et al. [8] proposed a lightweight and cost-effective sewer inspection framework for a floating capsule robot, utilizing computer vision techniques such as instance segmentation, real-time localization, and 3D reconstruction to achieve high-precision detection of fine-grained sewer defects. Nguyen et al. [9] presented a compact autonomous mobile robot designed for sewer pipeline inspection, capable of navigating small-diameter pipes using a simple, low-resource control strategy that enables complete network exploration without relying on visual data or high-energy image processing. Systems like CCTV inspection robots [10,11] and pipe-scanning robots [12,13] demonstrated high accuracy in defect recognition, allowing operators to assess pipeline conditions effectively. Although these robots excel at identifying problems, they are limited to inspection tasks and lack the ability to perform cleaning operations. This functional limitation necessitates additional tools or manual intervention for defect removal, increasing cleaning time and complexity.

The modular tooling approach has demonstrated success in industrial robotics. For instance, Mikolajczyk [14] achieved multi-process manufacturing through interchangeable tools on a single robotic platform. However, implementing such systems in sewers faces unique challenges. Many cleaning robots have been developed to address specific operational needs within sewers. For example, some robots were equipped with cutting tools designed for removing tree roots [15,16], while others focused on sediment removal using suction or scraping mechanisms [17,18]. Tugeumwolachot’s CIPbot-2 specializes in grinding operations, effectively cleaning hardened obstacles such as concrete deposits [19]. While these designs are highly effective in fulfilling their specialized cleaning tasks, the absence of visual systems limits their ability to provide real-time feedback on operational targets, making it challenging to accurately identify and locate obstacles such as roots, sediment, or debris during operation. Additionally, several commercially developed sewer robots are equipped with various cutting tools to clear foreign objects from pipelines, and they are fitted with rotatable cameras to provide real-time observation of operational targets [20,21,22]. Sulthana designed an automated robot capable of rapidly cleaning obstacles [23]. Awasthi introduced a robot equipped with an IR camera to relay real-time images of the sewage to a user-controlled mobile application. The user can then decide whether to intervene by directing the arm attached to the vehicle [24]. Due to the complexity of the sewer environment and the homogenization of images, these robots have difficulty performing cleaning tasks effectively when solely relying on manual operation.

This paper proposes a visually assisted Sewer Cleaning Robot (SCR), designed for efficient sewer cleaning. The mobile platform is equipped with a pressing mechanism, which improves stability during cleaning operation and enhances ability to cross obstacles within the complex sewer environment. The robot arm uses a linear actuator to replace rotational motors in the two pitch joints for increasing the load capacity. A Detection–Localization–Cleaning mode is propoesd for optimizing the cleaning process. This mode integrates the YOLO defect detection model and a monocular vision localization algorithm to enhance obstacle cleaning efficiency. By incorporating defect detection and target localization, SCR surpasses traditional robots that rely solely on visual feedback, enabling the quicker identification and localization of defects. It reduces the cognitive burden on operators and improves decision-making speed and accuracy. The functionality and effectiveness of SCR were validated through experiments conducted in simulated pipeline environments.

The remainder of this paper is structured as follows: Section 2 provides an overview of the hardware components of the SCR, Section 3 details the software algorithms employed, Section 4 presents the experimental results, and Section 5 offers concluding remarks.

2. Hardware System Design of SCR

The hardware system is designed to establish an independent and comprehensive operational platform within pipelines to address the complexities of the working environment. The hardware design of SCR integrates three core modules: a mobile platform, an operational arm, and a vision system. The mobile platform is equipped with a pressing mechanism designed to enhance the platform passability and ensure stability during cleaning operations. To meet the high-load requirements for cleaning hard obstacles, the pitch joints of the arm employ cost-effective and structurally stable linear actuator instead of motors. The overall structure of SCR is shown in Figure 1, and its main technical specifications are listed in

Table 1. Technical specifications of SCR.

Parameter	Value
Size	1210 mm
Weight	42 kg
Adaptable Pipe Radius	280–780 mm
Operation Length Range	50 m
Signal Type	485/CAN
Operation Method	Joystick, Keyboard
Turning Limitation	Limited to minor deviations

.

The robot measures 1210 mm in length, is constructed from stainless steel, and weighs 43 kg. It includes an automatic spooling device and a 50 m operating cable to facilitate extended, remote operation tasks. The remote control computer is powered by an Intel Core i5-1240P processor, equipped with 16 GB of RAM and an Intel Iris Xe graphics card, offering robust computational capabilities for real-time image transmission and human–computer interaction. Users can monitor and control the robot system in real-time through this computer, which also hosts all software algorithms and manages communication with each hardware module.

2.1. Hardware Mechanism Design

The arm is mounted at the front of the mobile platform, comprising the arm body, linear actuators, and a rotational motor. It provides three degrees of freedom (DoF), with a working range that covers any position within 780 mm sewer. The two pitch DoF of the arm are driven by a 400 mm stroke linear actuator using a double linear actuator structure, which directly delivers substantial torque. This setup is ideal for operations in confined spaces, enhancing arm stability and force control while also reducing costs. Double actuators ensure more stable movement and better load distribution than a single actuator, reducing potential problems such as tilting or uneven force application. At the end of arm, a high-speed cutting motor is equipped with various interchangeable cutting tools, enabling operations like cutting, chipping, and grinding to efficiently handle obstacles such as tree roots and cement slabs, ensuring both the feasibility and stability of operational tasks. During the process of maneuvering the arm to target, two DoF are controlled by linear actuators and combined with the arm mechanical structure, facilitating the pitching motion. The relationship between linear actuator length and joint angle is illustrated in Figure 2.

The blue triangle $\Delta a_1{b}_1{c}_1$ forms the principal triangle of the third DoF. In the initial state of the arm, the lifting arm angle ${\theta }_3=0$. $l_1$ denotes the length of the linear actuator. $m_1$ and $n_1$ represent the fixed lengths of the mechanical structure. When the lifting arm angle ${\theta }_3\ne 0$, based on the $\Delta a_1{b}_1{c}_1$, the relationship is described by Equation (1).

$$cos({\phi }_1+{\theta }_3)={\displaystyle \frac{{l_1}^2-{m_1}^2-{n_1}^2}{2m_1{n}_1}}$$

(1)

${\phi }_1+{\theta }_3$ represents the value of $\Delta a_1{b}_1{c}_1$ when the end-effector pitch angle is ${\theta }_3$; the length of the linear actuator $l_1$ is given by Equation (2).

$$l_1=\sqrt{2m_1{n}_{1}cos({\phi }_1+{\theta }_3)+{m_1}^2+{n_1}^2}$$

(2)

The red $\Delta a_2{b}_2{c}_2$ represents the principal triangle associated with the second DoF, which rotates around point $a_2$. In Figure 2, the arm is depicted in initial state, where the lifting arm angle ${\theta }_2=0$. $l_2$ denotes the length of the linear actuator, while $m_2$ and $n_2$ represent fixed-length structural components of the mechanical arm. When the lifting arm angle ${\theta }_2\ne 0$, based on the $\Delta a_2{b}_2{c}_2$, the relationship is described by Equation (3).

$$cos({\phi }_2+{\theta }_2)={\displaystyle \frac{{l_2}^2-{m_2}^2-{n_2}^2}{2m_2{n}_2}}$$

(3)

${\phi }_2+{\theta }_2$ represents the magnitude of ${\theta }_2\ne 0$ when the mechanical arm pitch angle $\angle c_2{a}_2{b}_2$. From Equation (3), the length of the linear actuator $l_2$ is given by Equation (4), when the lifting angle of the arm is ${\theta }_2$.

$$l_2=\sqrt{2m_2{n}_{2}cos({\phi }_2+{\theta }_2)+{m_2}^2+{n_2}^2}$$

(4)

According to the target pitch angle, the length of the linear actuator corresponding to the angle is calculated. Finally, the computer drives the controller to accurately reach the specified position.

The SCR vision system comprises front and rear gimbals, each fitted with a pinhole monocular camera (FS-ZK350, 1920 × 1080 pixels 25 fps) equipped with supplementary lighting. The system captures real-time images in a sewer through a 6 mm F1.4 fixed-focus lens (70° HFOV), facilitating the detection and localization of defects. The gimbals enable the camera to rotate vertically from 0 to 90 degrees. The image data collected by the vision system is transmitted to a ground workstation for real-time monitoring and operator decision-making, thus enabling comprehensive control over the robot operational status and the sewer environment.

2.2. Control System Hardware Parameters

The hardware control system of SCR is shown in Figure 3. The main controller collects video data from both the front and rear cameras and sends control commands to various subsystems. To address the slippery working conditions within sewers, an integrated servo hub motor with an IP65 waterproof rating is selected as the primary driving component. This motor is controlled by two 24 V dual closed-loop brushless DC servo drivers, and the drives receive speed control commands via the RS485 communication protocol. The pressing mechanism consists of two linear actuators, which are controlled by a single dual-synchronous driver. This driver also receives position control commands through the RS485 protocol. The rolling DoF for the arm is achieved using a 48 V hollow integrated motor, which provides a rolling range of ±180°. The motor receives angle control commands via the CAN communication protocol. The pitch DoF of arm employs an actuator drive controller identical to that used in the pressing mechanism.

The overall system uses five linear actuators. The end-effector uses a 48 V waterproof high-speed motor, delivering a continuous power of 1400 W and reaching a maximum speed of 3000 rpm. This motor is regulated by a DC chopper PWM three-phase driver with a peak current of 30 A. Finally, the camera gimbal is controlled by a 24 V DC servo motor, which receives position control commands through the RS485 protocol, enabling the precise directional adjustments of the camera.

3. Software Algorithm Design

Robot perception and decision-making capabilities are insufficient for autonomously performing cleaning tasks in complex pipeline environments. With the assistance of machine vision, combined with human decision-making expertise, the efficiency and accuracy of remote operations can be significantly improved. To address the issue of users being unable to quickly obtain target tasks in the presence of various defects within the pipeline, this paper integrates the YOLOv5 defect detection model with a monocular vision localization method in the visual system. This system can identify the types of defects in the pipeline in real-time and calculate their locations through the monocular vision localization method, which utilizes two images. All algorithms are designed within the Robot Operating System (ROS) framework, with communication between functional modules occurring through custom messages sent via ROS topics and services.

Figure 4 illustrates the robot environmental perception and task flowchart. The robot is deployed into the pipeline via a hoisting mechanism. The operator selects the type of end-effector currently installed and then controls the movement of the mobile platform. During the movement, the defect detection module operates automatically. When a defect is identified, if it matches the current tool type, its location is calculated. The current robot uses a single-purpose end-effector for each task. When encountering different types of obstacles, the operator retrieves the robot to change the end-effector tool. In the absence of external factors such as slippage, the robot estimates its position within the pipeline by utilizing the drive wheel speed sensors and the visual system. Once the robot identifies and locates a task target, the operator controls the mobile platform to move to the work area. Based on the type of target, it is determined whether the pressing mechanism needs to be raised. Finally, the arm is manually controlled to perform the cleaning task.

The control interface is shown in Figure 5. The first part of Figure 5 is the camera feedback image, which displays the information transmitted from the front and rear cameras of the robot. The second part contains the control interface of the robot body and arm, including the adjustment of the hub motor speed and the linear actuator length. The third part is the keyboard operation area, which provides the same control functions as the second part. The fourth part displays the feedback information from the arm rolling joint motor driver.

3.1. Defect Detection Modeling in Sewers

To identify different obstacles such as tree roots and slats within sewers and to assist operators in non-excavation removal operations, SCR requires a target detection model with high accuracy, rapid detection speed, and strong generalizability to adapt to varying sewer environments. In recent years, YOLO series algorithms have demonstrated advantages in the field of target detection, including fast processing speeds, high accuracy, and robust generalization. After optimization, these algorithms can be effectively applied to defect detection in sewers.

In this study, we employ the YOLOv5-sewer model [25], trained using the Storm Drain Model Dataset [26] and Pipe Root Dataset [27] from the Robotflow Universe open-source dataset platform. The dataset consists of 1317 images featuring various target defects, such as cracks, utility intrusions, obstacles, and tree roots, with a total of 3034 labeled defects. Examples of the dataset images are shown in Figure 6.

3.2. Monocular Visual Localization

The environment inside sewer is characterized by complexity, homogeneity, and dim lighting over medium and long distances. Additionally, pipe defects exhibit both visual diversity and similarity [28], which can lead to reduced accuracy in defect detection. To address the challenge of portable equipment at operation sites being unable to run real-time defect detection models with high accuracy, a lightweight sewer defect detection model, YOLOv5-Sewer, optimized for pipeline environments, is employed. This model achieves a lightweight design that enhances its performance on low-resource devices while maintaining detection accuracy [25].

To enable efficient obstacle cleaning by SCR and to provide operators with accurate target location information, target localization plays a critical role in the overall cleaning operation process. Target localization involves utilizing the target feature information from the image plane to obtain the position in 3D space. Monocular vision localization is a technique that uses a single camera to estimate the position of objects in a 3D space. Compared with binocular vision systems, monocular vision systems require only one camera, offering advantages such as a simpler structure, a lower cost, greater flexibility, and convenience [29]. Additionally, monocular vision eliminates the need to consider factors like the optimal baseline distance between two cameras and blind zones, making it a more practical and versatile solution. These features provide monocular vision systems with broader application prospects in real-world scenarios.

This paper uses a monocular visual localization method that utilizes edge detection within defect bounding boxes across two images to achieve precise target positioning. The workflow is depicted in Figure 7. First, the camera is calibrated to determine its intrinsic matrix K and distortion coefficient D:

$$K=\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]$$

(5)

$$D=\left[\begin{array}{ccccc}{k}_1& k_2& p_1& p_2& k_3\end{array}\right]$$

(6)

The intrinsic matrix K encodes the camera internal parameters, which include the focal lengths $f_x$ and $f_y$ along the x and y axes, and the coordinates of the principal point $(c_x,c_y)$, which is usually near the center of the image. The intrinsic matrix is essential for converting 3D points in the camera coordinate system into 2D pixel coordinates in the image. The distortion coefficients D account for lens distortion and include both radial and tangential distortion parameters. Specifically, $k_1$, $k_2$, and $k_3$ are the radial distortion coefficients, while $p_1$ and $p_2$ are the tangential distortion coefficients. Radial distortion causes straight lines to appear curved, commonly observed as barrel or pincushion distortion, while tangential distortion arises due to lens assembly imperfections.

These parameters are determined during the camera calibration process using a series of images of a known calibration pattern, such as a checkerboard. Using the OpenCV library cv2.calibrateCamera function, the intrinsic matrix K and the distortion coefficients D are computed simultaneously.

The camera is subsequently repositioned to capture two images of the scene from distinct viewpoints, ensuring sufficient parallax for accurate depth estimation. A defect detection model is utilized to process the images, identify the target defects, and generate bounding boxes around the areas of defect. Within these bounding boxes, the Canny edge detection algorithm is applied to enhance the edges and extract the structural features of the defect.

Canny edge detection operates by analyzing the image gradients to identify areas of rapid intensity changes, which typically correspond to edges. The gradient magnitude (G) and direction ($\theta$) are computed using the following equations:

$$G=\sqrt{G_x^2+G_y^2},\theta =arctan\left({\displaystyle \frac{G_y}{G_x}}\right)$$

(7)

In Equations (7), $G_x$ and $G_y$ represent the horizontal and vertical gradients of the image, respectively. The gradient magnitude G reflects the strength of the edge at each pixel, while the gradient direction $\theta$ indicates the orientation of the edge relative to the horizontal axis.

By isolating the edges within the defect bounding boxes, this approach reduces the computational complexity and enhances the precision of subsequent monocular vision localization. These extracted edges serve as key features for matching across the two viewpoints, enabling accurate depth estimation and precise positioning of the defect within the pipeline environment.

The detected edge information is then restored to its original position in the image. ORB feature detection and descriptor computation are performed on the edge-detected image, and feature matching is carried out using the Hamming distance:

$$H({\mathbf{d}}_1,{\mathbf{d}}_2)={\displaystyle \sum _{i=1}^n}\left|{\mathbf{d}}_1\left[i\right]-{\mathbf{d}}_2\left[i\right]\right|$$

(8)

where ${\mathbf{d}}_1\left[i\right]$ and ${\mathbf{d}}_2\left[i\right]$ are the i-th bits of the respective descriptors, and n is the length of the descriptor. The Hamming distance measures the number of bit positions where the two descriptors differ, effectively quantifying their dissimilarity.

To optimize the results, the RANSAC algorithm is used to filter out noise and incorrect matches. The RANSAC algorithm iteratively selects subsets of feature matches to estimate the fundamental matrix F while reducing outliers caused by incorrect feature correspondences. The fundamental matrix F is a $3\times 3$ matrix that encapsulates the epipolar geometry relationship between two views. It satisfies the following constraint for corresponding points $\mathbf{x}$ and ${\mathbf{x}}^{\prime }$ in the two images:

$${\mathbf{x}}^{\prime \top }F\mathbf{x}=0$$

(9)

In Equations (9), $\mathbf{x}$ and ${\mathbf{x}}^{\prime }$ are the homogeneous coordinates of a point in the first and second images, respectively. The constraint ensures that point ${\mathbf{x}}^{\prime }$ lies on the epipolar line corresponding to $\mathbf{x}$.

Compared to direct ORB feature matching applied only within the bounding box, integrating Canny edge detection focuses feature points specifically on the target edge regions. This selective approach minimizes interference from irrelevant background details, which is particularly beneficial in complex pipeline scenarios with repetitive textures or occlusions. By concentrating feature points along the object boundaries, the matching process achieves improved accuracy and computational efficiency.

After computing the fundamental matrix F using the RANSAC algorithm, the essential matrix E can be derived by incorporating the camera intrinsic matrix K:

$$E=K^{\top }FK$$

(10)

The essential matrix E encodes the relative rotation and translation between two camera views, and it forms the foundation for recovering the camera motion.

To extract the rotation matrix R and translation vector $\mathbf{t}$ from the essential matrix E, singular value decomposition (SVD) is performed. The essential matrix E is decomposed as follows:

$$E=U·\Sigma ·V^{\top }$$

(11)

where U and V are orthogonal matrices, and $\Sigma =\mathrm{diag}(s,s,0)$ is the singular value matrix, with s representing the non-zero singular values of E. Based on this decomposition, two possible rotation matrices, $R_1$ and $R_2$, and two possible translation vectors, $\pm \mathbf{t}$, can be obtained. To determine the correct combination of R and $\mathbf{t}$, a triangulation step is performed, ensuring that the reconstructed 3D points lie in front of both camera planes. This step selects the valid solution among the four possible configurations $(R_1,\mathbf{t})$, $(R_1,-\mathbf{t})$, $(R_2,\mathbf{t})$, and $(R_2,-\mathbf{t})$.

Finally, using the known camera pose and the triangulation method [30], the 2D image points are reconstructed into 3D coordinates. The relationship is defined by the following equations:

$${\lambda }_1{\mathbf{x}}_1=K\left[I\right|\mathbf{0}]\mathbf{X},{\lambda }_2{\mathbf{x}}_2=K\left[R\right|\mathbf{t}]\mathbf{X}$$

(12)

In Equation (12), ${\lambda }_1$ and ${\lambda }_2$ are scale factors that convert the homogeneous coordinates into Cartesian coordinates, while ${\mathbf{x}}_1$ and ${\mathbf{x}}_2$ represent the 2D pixel coordinates of the corresponding point in the first and second images, respectively. The camera intrinsic matrix M contains parameters such as the focal length and principal point offsets. The pose of the first camera is expressed as $\left[I\right|\mathbf{0}]$, assuming it is located at the origin of the world coordinate system, whereas $\left[R\right|\mathbf{t}]$ describes the rotation and translation of the second camera relative to the first. Finally, $\mathbf{X}$ denotes the 3D point in the world coordinate system.

These equations form the basis of triangulation, where the intersection of two rays determines the 3D location of the target. By applying this process to all valid matches, a 3D reconstruction of the identified defects is achieved, enabling precise localization and subsequent cleaning operations.

3.3. 3-DoF Arm Kinematic Analysis

The arm has three DoF, with a cutting motor rotary joint serving as the end-effector. The kinematic model of the arm is depicted in Figure 8.

The rotational ranges of the joints $J_1$, $J_2$, and $J_3$ are $-{180}^{\circ }$ to ${180}^{\circ }$, $0^{\circ }$ to ${40}^{\circ }$, and $0^{\circ }$ to ${90}^{\circ }$, respectively. Using the Denavit–Hartenberg (D-H) parametric method, the kinematic equations for the arm are derived and expressed in Equation (13). The lengths of $l_1$, $l_2$, and $l_3$ and are 0 mm, 358 mm, and 147 mm, respectively.

$${}_4{}^{0}T=\left[\begin{array}{cccc}c{\theta }_{1}c({\theta }_2+{\theta }_3)& -c{\theta }_{1}s({\theta }_2+{\theta }_3)& 0& l_{3}c{\theta }_{1}c({\theta }_2+{\theta }_3)+l_{2}c{\theta }_{1}c{\theta }_2\\ s{\theta }_{1}c({\theta }_2+{\theta }_3)& -s{\theta }_{1}s({\theta }_2+{\theta }_3)& 0& l_{3}s{\theta }_{1}c({\theta }_2+{\theta }_3)+l_{2}s{\theta }_{1}c{\theta }_2\\ s({\theta }_2+{\theta }_3)& c({\theta }_2+{\theta }_3)& 0& l_{3}s({\theta }_2+{\theta }_3)+l_{2}s{\theta }_2\\ 0& 0& 0& 1\end{array}\right]$$

(13)

When the parameters in the ${}_4{}^{0}T$ are known, they are expressed as Equation (14).

$${}_4{}^{0}T=\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& p_x\\ r_{21}& r_{22}& r_{23}& p_y\\ r_{31}& r_{32}& r_{33}& p_z\\ 0& 0& 0& 1\end{array}\right]$$

(14)

This can be solved for ${\theta }_1$, ${\theta }_2$, and ${\theta }_3$, as shown in Equations (15)–(17).

$${\theta }_1=arccos{\displaystyle \frac{r_{11}}{r_{32}}}=arcsin{\displaystyle \frac{r_{21}}{r_{32}}}$$

(15)

$${\theta }_2=arcsin{\displaystyle \frac{p_z-l_3{r}_{31}}{l_2}}=arccos{\displaystyle \frac{p_x-l_{3}c{\theta }_1{r}_{32}}{l_{2}c{\theta }_1}}$$

(16)

$${\theta }_3=arccos(s{\theta }_2{r}_{31}+c{\theta }_2{r}_{32})=arc\tan {\displaystyle \frac{r_{31}}{r_{32}}}-{\theta }_2$$

(17)

4. Results of Experiments and Discussions

In this section, we conduct experiments to verify the feasibility of the SCR in obstacle cleaning tasks. These experiments include testing the obstacle-crossing capability of the mobile platform, performing monocular vision-based defect detection and localization, and obstacle cleaning operations. The results demonstrate both the rationality of the robot design and its feasibility in executing obstacle cleaning tasks.

4.1. Mobile Platform Crossing Test

To verify the robot obstacle-crossing ability in sewers, an average of 20 mm thick branches and leaves were placed in a simulated pipeline to replicate the soft obstacles typically found in such environments. Figure 9 shows the mobile platform accelerating through the soft obstacle with the pressing mechanism lifted, with arrows indicating the platform direction of movement. The test results demonstrate that the mobile platform has good passability on soft terrain.

4.2. Monocular Vision Defect Detection and Localization Experiment

In the monocular vision defect detection experiment, defects from the Pipeline Computer Vision Project dataset were identified using the YOLOv5-Sewer lightweight model, which has been optimized for in-pipe environments. The detection results are illustrated in Figure 10. All six pipeline defects within the dataset were successfully recognized, demonstrating effective detection performance even in complex environments.

In the target localization experiments, the camera is initially calibrated to determine its intrinsic parameters and distortion coefficients. The calibration was performed using the OpenCV library calibration function. The resulting intrinsic matrix is presented as Equation (18), and the distortion coefficients are given in Equation (19).

$$K_0=\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]=\left[\begin{array}{ccc}579.9844& 0& 315.8443\\ 0& 582.27447& 240.6151\\ 0& 0& 1\end{array}\right]$$

(18)

$$D_0=\left[\begin{array}{ccccc}{k}_1& k_2& p_1& p_2& k_3\end{array}\right]=\left[\begin{array}{ccccc}0.15539& -0.38861& 0.00446& 6.574\times {10}^{-4}& 0.29998\end{array}\right]$$

(19)

Figure 11 represents the ORB feature point matching of two images taken at different moments. The RANSAC algorithm is used to compute the fundamental matrix, and the wrong matching points are automatically eliminated. Fundamental matrix is solved as Equation (20).

$$F_1=\left[\begin{array}{ccc}0& 0& 0.0044\\ 0& 0& -0.0142\\ -0.0068& 0.0096& 1\end{array}\right]$$

(20)

Using the camera intrinsic matrix $K_0$ and the fundamental matrix $F_1$ from Equation, the essential matrix $E_1$ can be derived as shown in Equation (21).

$$E_1=\left[\begin{array}{ccc}0.506& -14.761& -3.265\\ 15.982& -0.285& 0.305\\ 2.901& -2.537& -0.496\end{array}\right]$$

(21)

The essential matrix is decomposed using the cv2.recoverPose function to obtain the rotation matrix $R_1$ and translation vector $t_1$, as represented by Equation (22) and Equation (23), respectively.

$$R_1=\left[\begin{array}{ccc}0.9899& 0.0262& -0.1391\\ -0.0201& 0.9987& 0.0458\\ 0.1401& -0.0426& 0.9892\end{array}\right]$$

(22)

$$t_1=\left[\begin{array}{ccc}0.1636& 0.1712& -0.9715\end{array}\right]$$

(23)

The projection matrix for the two frames is constructed using the camera intrinsic parameters and the positional transformation data. Subsequently, triangulation is employed to determine the target depth in the current camera coordinate system, calculated to be 44.3 cm, with the actual distance being 45 cm. Multiple localization calculations were conducted by varying the distance. The experimental results are depicted in Figure 12. The monocular visual localization system demonstrates good performance at short to medium distances, achieving the highest accuracy at 75 cm.

However, the error increases as the distance grows. Measurement accuracy at longer distances requires further optimization. Additionally, the impact of various environmental factors on measurement accuracy must be considered in practical applications.

4.3. In-Pipe Obstacle Cleaning Experiments

Branches and bricks were fixed at the end of the pipeline to simulate the cutting of tree roots and the grinding of hard obstacles typically encountered in sewers. The operator remotely controlled the robot to perform cutting and grinding operations in the simulated pipeline. The experimental process is shown in Figure 13 and Figure 14. After elevating the pressing mechanism, the mobile platform was maneuvered in the direction indicated by the yellow arrow. In the tree root cutting experiment, the red box denotes the target operational area. Upon reaching this area, the arm was maneuvered to execute the cutting of the branches. In the brick grinding experiment, the end-effector is replaced with a special grinding tool. The area in the yellow box in the Figure 14 indicates that the grinding operation is in progress, and the mobile platform is controlled to move forward to perform the grinding operation.

Figure 15 illustrates the cross-sections of the cut tree root and the sanded brick surface. The cross-section of the cut tree root is neat, and the thickness of the polished brick surface is reduced by 1 cm. This demonstrates that the current end effector can complete the cutting and grinding tasks in the sewer. However, in the grinding experiment, a deviation was observed when the arm made contact with the bricks surface, potentially leading to unintended contact with the pipe wall.

5. Conclusions

This paper proposes the development and validation of an SCR, a visually assisted robotic system designed for cleaning tasks in narrow and complex sewers. The key contributions include the following:

• Key Features:

  –

  A mobile platform with a pressing mechanism to enhance stability and obstacle-crossing capabilities.

  –

  The arm uses cost-effective, structurally stable linear actuators in pitch joints to meet high-load requirements and ensure operational stability.

  –

  The integration of the YOLO defect detection model and a monocular vision localization algorithm for a Detection–Localization–Cleaning mode.

• Advantages:

  –

  Enhances cleaning precision and efficiency through defect detection and target localization algorithms.

  –

  Reduces operator burden and improves decision-making speed in complex pipeline scenarios.

The feasibility and effectiveness of the SCR were validated through simulated experiments, which demonstrated its mobility, visual detection and localization accuracy, and obstacle cleaning capabilities. These results confirm the rationality of the robot design and its potential for practical applications in sewer cleaning tasks.

This work provides technical reserves for the further development of vision assisted cleaning robots in the field of sewer cleaning. Future research will focus on enhancing the robustness of the visual algorithm and the stability during cleaning operations, thereby improving its adaptability to different sewer environments.