Trans-DCN: A High-Efficiency and Adaptive Deep Network for Bridge Cable Surface Defect Segmentation

Abstract

Cables are vital load-bearing components of cable-stayed bridges. Surface defects can lead to internal corrosion and fracturing, significantly impacting the stability of the bridge structure. The detection of surface defects from bridge cable images faces numerous challenges, including shadow disturbances due to uneven lighting and difficulties in addressing multiscale defect features. To address these challenges, this paper proposes a novel and cost-effective deep learning segmentation network, named Trans-DCN, to detect defects in the surface of the bridge cable. The network leverages an efficient Transformer-based encoder and integrates multiscale features to overcome the limitations associated with local feature inadequacy. The decoder implements an atrous Deformable Convolution (DCN) pyramid and dynamically fuses low-level feature information to perceive the complex distribution of defects. The effectiveness of Trans-DCN is evaluated by comparing it with state-of-the-art segmentation baseline models using a dataset comprising cable bridge defect images. Experimental results demonstrate that our network outperforms the state-of-the-art network SegFormer, achieving a 27.1% reduction in GFLOPs, a 1.2% increase in mean Intersection over Union, and a 1.5% increase in the F1 score. Ablation experiments confirmed the effectiveness of each module within our network, further substantiating the significant validity and advantages of Trans-DCN in the task of bridge cable defect segmentation. The network proposed in this paper provides an effective solution for downstream cable bridge image analysis.

1. Introduction

Bridge cables have undergone significant technological advancements, evolving from iron chains to modern high-strength steel cables. These cables play a crucial role in expanding bridge spans and ensuring structural stability and safety, serving as essential load-bearing components in suspension bridges and cable-stayed bridges. However, due to exposure to the harsh natural environment, high-intensity load, and material corrosion, bridge cables are susceptible to various defects, including cracks, potholes, scratches, etc. The surface defects may result in the rapid deterioration of the bridge cables, posing significant threats to bridge safety [1]. Therefore, timely monitoring of the structural health of the bridge cables is imperative to inform appropriate maintenance interventions [2].

The traditional methods for detecting bridge cable defects relied solely on manual visual inspection and basic physical measurement tools, such as laser scanning. Manual inspection is extremely labor-intensive and inefficient, particularly when confronted with numerous extreme working environments. With the progression of science and technology, certain physical technical applications, such as ultrasonic detection methods and optical fiber sensing technology, have gradually been implemented [3,4]. However, these instruments tend to be costly and are more suited for detecting internal defects, making it challenging to observe and evaluate surface integrity. The potential incompleteness in surface inspection poses a hazard, as these surface defects can rapidly lead to internal corrosion and fractures. In recent years, computer vision technology and artificial intelligence algorithms have demonstrated their efficacy in various engineering domains and have been extensively employed in bridge cable defect detection.

Cable-climbing robots have become prevalent in inspecting bridge cable defects due to their efficient data acquisition capabilities and stability. Currently, numerous studies have explored the integration of cable-climbing robots with computer vision systems for bridge defect detection tasks [5,6,7]. Leveraging the widespread availability and convenience of these cable-climbing robots, we employed an embedded system-based vision acquisition device for cable defect detection. This device comprises four RGB cameras arranged around a central cable, a pulley power system, an image acquisition system, and a wireless control system, as shown in Figure 1. The robot can adapt to cables of various sizes and capture exterior images from four perspectives.

One of the primary challenges in bridge cable surface inspection lies in the effectively utilizing collected imagery to rapidly identify the precise locations of defects and damage details. Current research has integrated cable-climbing robots with computer vision for application in bridge cable defect detection tasks, encompassing both classical computer vision algorithms and advanced deep learning methodologies for detection purposes. The former often require significant manual intervention for preprocessing and struggle to cope with the challenges posed by lighting variations. The latter approach reduces the preprocessing stage but demands a large volume of image samples [8]. Moreover, the conventional network framework lacks generalization ability and robustness when confronted with changes in lighting and complex distributions of defects [9].

We identify two primary challenges in enhancing the generalization capability and robustness of deep neural networks for bridge cable defect extraction. The first challenge arises from constantly changing collection environments, such as insufficient lighting or angles obstructed by shadows, as well as complex background texture interference other than the cable itself. This can lead to the omission of local features and incorrect associations. It is crucial to fully extract effective feature information from the images. Transformers, based on the self-attention mechanism, establish direct global relationships among all pixels in the image, effectively capturing and utilizing long-range dependency information [10,11,12], thus effectively addressing this challenge. Therefore, in our proposed deep neural network, we introduce a novel Transformer as the encoder to comprehensively extract global contextual feature relationships, compensating for the lack or absence of local features, a capability not inherent in convolutional methods based on local spatial aggregation [13]. Additionally, we integrate feature information from all stages, retaining potential defect-related features to precisely delineate defect areas according to the requirements in the decoder.

Another challenge arises from the intricate and varied genre of bridge cable defects, mainly reflected in the fact that the same defect may appear in multiple sizes and the defects exhibit a broad spectrum of changes and diverse textures. Particularly, cracks typically appear as thin dark lines or strips with constantly varying angle and direction, such as circumferential cracks, longitudinal cracks, and minor cracks. Standard convolutions struggle to adequately capture such elongated features, as different positions may correspond to defects of different scales or deformations [14]. Fixed spatial aggregation methods may confuse the encoding of positional information for deep networks. Therefore, it is imperative to comprehensively extract these defect regions. We proposed a Serial-Parallel pyramid module based on atrous Deformable Convolution (SPP-DCN) in the decoder, employing multiple progressively dilated Deformable Convolutions to adaptively capture the distribution of defect features. This is followed by the utilization of the Squeeze-and-Excitation (SE) mechanism to accentuate effective boundary information. Meanwhile, within the low-level layers of the encoder, which capture rich edge information at the local level, we introduce several shortcuts between the encoder and the decoder. An efficient Adaptive Spatial Feature Fusion (ASFF) module is employed to comprehensively integrate and summarize this edge information in the decoder, ensuring the integrity of defect extraction.

RGB natural images are a common data format in the field of defect detection. Consequently, our study focuses on developing a deep learning network for bridge cable defect detection based on RGB data input. Traditional segmentation frameworks often employ a simple encoder–decoder structure to extract features, followed by direct deconvolution or upsampling to obtain segmentation results. However, this approach is not well suited for the specific characteristics of cable defect detection. To achieve effective, rapid, and robust detection of surface defects in bridge cables and address the aforementioned challenges, we propose an end-to-end efficient semantic segmentation deep neural network named Trans-DCN. The Trans-DCN model features an efficient feature extraction encoder and a fully refined feature-aware decoder, which are crucial for bridge cable defect segmentation. The code is publicly available at https://github.com/hzh1231/Trans_dcn (accessed on 21 July 2024).

Thus, the primary contributions of this paper are as follows:

• The introduction of an efficient backbone based on Transformer, complemented by a multi-layer feature aggregation module in the encoder, facilitating the comprehensive utilization of global contextual defect features.
• The establishment of a serial–parallel structure featuring multiple atrous Deformable Convolutions, which dynamically adjust the receptive field according to the defect distribution, ensuring a comprehensive detection with multiple granularities.

In this paper, we specifically illustrated the technical details of the network and conducted experiments comparing it with state-of-the-art models. We also analyzed and discussed the effectiveness of each module in our work to demonstrate the new baseline performance in bridge cable defect detection.

2. Related Work

2.1. Bridge Cable Surface Defect Detection

Polyethylene (PE) pipes serve as the protective layer for bridge cables. Once the PE pipe ages or ruptures, corrosive substances can infiltrate the internal steel wires through the defect in the protective layer [15]. Therefore, it is crucial to assess the surface defect of the cables non-destructively [16]. Figure 2 illustrates some common surface defects found in bridge cables.

Traditional manual inspection methods are inefficient and subjective, making it difficult to meet the maintenance needs of modern bridges. Numerous researchers have explored the utilization of cameras mounted on Unmanned Aerial Vehicles (UAVs) or cable-climbing robots to capture images of surface defects on bridges. These images are then subjected to intelligent analysis through computer vision techniques [7,17,18,19]. Threshold-based methods, which rely on the analysis of grayscale and gradient features of images, can achieve basic defect extraction [20,21]. However, directly applying image processing methods for defect detection encounters challenges regarding insufficient robustness, often necessitating extensive preprocessing to attain satisfactory results. With further technological advancements, machine learning, by constructing high-level feature representations, can effectively address the challenges posed by changes in lighting conditions [22]. Li et al. achieved high accuracy in defect image classification experiments by deploying the Support Vector Machine (SVM) based on particle swarm optimization [1]. Nonetheless, feature extraction for defect images still required the use of mathematical statistical methods.

Typically, deep learning has proven effective in detecting defects in domains with similar distributions [23]; some research has already achieved basic defect detection by utilizing simple and common network architectures [24,25,26]. However, these generic framework methods struggle to adapt to the specific characteristics of cable images, particularly when facing predominantly strip-like cracks, which often result in missed or false detections of defects. Therefore, we have developed an advanced approach for defect detection that takes into account environmental disturbances and the distribution characteristics of defects, with the aim of fully enhancing the potential of neural network applications.

2.2. Image Segmentation Based on Deep Neural Network

Most previous studies have proposed methods based on image classification or object detection using bounding boxes [27,28,29]. However, these methods cannot provide precise information regarding defect paths and densities. Therefore, there is a pressing demand for a pixel-based segmentation approach to distinguish defective pixels.

With sufficient training data, supervised learning strategies significantly enhance the capabilities of segmentation models. The Convolutional Neural Network (CNN) framework is the most commonly used approach, as CNNs demonstrate powerful feature extraction capabilities, especially in capturing local features within images [13,30,31,32]. In particular, Long et al. initially introduced the Fully Convolutional Network (FCN), which can be trained end-to-end, greatly improving accuracy and paving the way for deep learning-based pixel-level semantic segmentation [33]. This advancement assists intelligent systems in grasping spatial relationships or making critical judgments [34]. Shi et al. segmented cracks in steel and rubber bearing corrosion images using VGG-Unet [35]. Despite being affected by the imbalance between foreground and background in the dataset, this encoder–decoder framework was considered one of the most well-designed architectures for segmentation, as it is well suited for recovering multiscale details of target objects [36]. Deng et al. incorporated the Atrous Spatial Pyramid Pooling (ASPP) module into LinkNet, achieved high recognition accuracy for bridge surface structural damage based on a pre-trained encoder, even on small datasets [37]. The network proposed is based on an encoder–decoder framework and utilizes transfer learning from large datasets to quickly deploy into our small domain of defect segmentation. Additionally, composite loss functions were employed to better learn foreground defect features.

Although convolutional networks demonstrate powerful capabilities in extracting local features, their fixed receptive fields limit the ability to correlate features at large scales. In contrast, self-attention mechanisms enrich the representation capability of features based on global texture modeling, effectively compensating for the limitations of convolutions. Self-attention has recently gained favor among many researchers after Transformers became the new state of the art in Natural Language Processing (NLP) [12]. This led to the development of Vision Transformer (ViT) [11], which has been adapted for image segmentation tasks [38,39,40]. The Swin Transformer confines self-attention within sliding windows, significantly reducing computational costs [41]. He et al. embedded the Swin Transformer into UNet to form a dual encoder, which is used in remote sensing image segmentation, enhancing the accuracy of small-scale ground objects through a spatial interaction module and a feature compression module [42]. However, the increased computational cost is not conducive.

Our proposed model effectively integrates Transformers and convolutional networks. The efficient encoder, developed based on the Transformer, comprehensively extracts potential defect features, while Spatial Reduction Attention (SRA) is introduced to reduce complexity [43]. Subsequently, in the convolution-based decoder, features extracted by the encoder are further refined, ultimately yielding critical bridge cable defect predictions.

3. Method

We propose an encoder–decoder defect image segmentation architecture, Trans-DCN, designed to efficiently achieve comprehensive detection of the defect areas of bridge cables. In this section, we will outline our overall network architecture. The flowchart of Trans-DCN is shown in Figure 3. Before that, it is necessary to provide a brief overview of some relevant techniques used in the network, including Dot Product Self-Attention (DPSA), Deformable Convolution (DCN), Depthwise Separable Convolution (DSC), and SE.

3.1. Overview of Related Methods

Self-attention mechanism. In a self-attention [12], given an input sequence $s\in {\mathbb{R}}^{N\times d}$, we obtain $\mathit{Q}$(query), $\mathit{K}$(key) and $\mathit{V}$(value) $\in {\mathbb{R}}^{N\times D}$ matrices with different spatial relationships by multiplying three learnable mapping matrices $W_q$, $W_k$, $W_v\in {\mathbb{R}}^{d\times D}$ with s. The Dot Product Self-Attention is defined by scaling the dot product of $\mathit{Q}$ and $\mathit{K}$ and passing it through a softmax activation to produce attention weights, which are then applied to the $\mathit{V}$, represented as follows:

$$Attention(Q,K,V)=softmax\left({\displaystyle \frac{Q·K^T}{\sqrt{D}}}\right)V$$

(1)

where $\sqrt{D}$ represents the scaling parameter, and N is the length of the embedding sequence. Self-attention can be regarded as the weighted average of the input sequence with respect to itself and all other sequences. It aggregates information from all sequences, thereby extracting rich contextual information from the image. Meanwhile, the computational complexity of self-attention is quadratic, $\mathcal{O}\left(N^{2}D\right)$, primarily dependent on the number of input sequences.

The self-attention mechanism has been widely utilized in the field of NLP. ViT is one of the earliest works to apply a pure Transformer to computer vision and demonstrates a powerful large-scale correlation capability of image features, a degree of correlation that is challenging for convolution-based networks to achieve. Given the large-scale variations observed in defect areas of bridge cables, the broad-contextual feature correlation enabled by self-attention can enhance the network’s capability to extract relevant information. Therefore, we proposed a multiscale feature alignment-utilizing backbone to serve as the encoder of the model.

Deformable Convolution. For regular 2D convolution in the field of image processing, we first define a receptive field grid R, which represents the deviation from the center of the convolution kernel. Taking a convolutional kernel $3\times 3$ as an example, R with one dilation is as follows: $R=\left\{\right(-1,-1),(-1,0),\cdots ,(1,1\left)\right\}$. For a given input feature map x, after convolutional computation, each element $p_o$ on the output feature map y can be obtained through the following calculation:

$$y\left(p_o\right)=\sum _{p_n\in R}w\left(p_n\right)·x\left(p_o+p_n\right)$$

(2)

where $q_n$ enumerates the elements of R, and w is the learnable weight. It is worth noting that bias has not been considered in this formulation.

Compared to attention mechanisms, regular convolution exhibits lower computational complexity but is constrained by certain spatial aggregation capabilities, thereby sacrificing the ability to aggregate broad-contextual information. Deformable Convolution seeks to strike a balance between self-attention and regular convolution by introducing additional learnable offsets [14], as shown in Figure 4a. In Deformable Convolution, the external offset $\Delta p_n$ is added to the receptive field grid R. Therefore, Deformable Convolution is represented in Equation (2) as follows:

$$y\left(p_o\right)=\sum _{p_n\in R}w\left(p_n\right)·x\left(p_o+p_n+\Delta p_n\right)$$

(3)

where $\Delta p_n$ is learnable from the input feature map and maintains the same number of R. Particularly, the learned $\Delta p_n$ are floating-point numbers, while the feature map is integer. Bilinear interpolation sampling is required to obtain $x(p_o+p_n+\Delta p_n)$.

Deformable Convolution inherits the efficient capability of regular convolution while also possessing adaptive spatial aggregation capability and dynamic adjustment of the range of perceived features, which is similar to self-attention. We propose a serial–parallel Deformable Convolution module in the decoding stage, which achieves adaptive multiscale spatial feature aggregation and ensures a comprehensive perception of the defects.

Depthwise Separable Convolution. Depthwise Separable Convolution was first introduced in MobileNet [44]. It factorizes a standard convolution into a depthwise convolution and a pointwise convolution. Specifically, each input feature channel corresponds to a unique kernel in depthwise convolution, ensuring that the convolution operates independently on each channel. Subsequently, information from different channels is linearly combined into a fixed number of channels in $1\times 1$ pointwise convolution, facilitating efficient information exchange and integration between channels, as shown in Figure 4b.

In addition, we also incorporate batch normalization and ReLU activation after each layer, enhancing the convolutional operations’ fitting capabilities [45]. Depthwise Separable Convolution is more efficient than standard convolution. Given the kernel size K and output channel size N, we can obtain the ratio of computational complexity to standard convolution as ${\displaystyle \frac{1}{N}}+{\displaystyle \frac{1}{K^2}}$. Therefore, we utilize DSC as a substitute for standard convolution to reduce computational costs in our model.

Squeeze-and-Excitation. The SE introduces compression (Squeeze) and excitation (Excitation) operations to establish relationships between channels, enabling the model to adaptively learn the importance of each channel and adjust its contribution in the feature map [46]. Therefore, the model can focus more on important channels, enhancing the discriminative capability of target feature. The SE mechanism can be represented as follows:

$$SE\left(F\right)=F⨂sigmod\left(FC\left(ReLU\left(FC\left(GP\left(F\right)\right)\right)\right)\right)$$

(4)

where F is the input feature, ⨂ denotes the element-wise multiplication operation, $FC$ represents the fully connected layer, $GP$ stands for the global pooling layer, and $sigmod$ and $ReLU$ are non-linear activation layers.

3.2. Proposed Network Architecture

In this paper, we propose a state-of-the-art segmentation model Trans-DCN (Figure 5) that combines the effective feature extraction capability of Transformers as the encoder (Section 3.3), and integrates the adaptive aggregation structure as the decoder (Section 3.4). By establishing an encoder–decoder framework that combines the advantages of self-attention and convolution, we aim to establish a baseline that could accurately identify defect locations on bridge cables.

In the encoder, we adopt the fundamental Transformer encoding paradigm [11,41,43] to capture global contextual relationships. Our proposed Transformer encoder comprises four stages without positional encoding (The positional information is contained in the FNN module subsequently). Preceding each encoder, we include a patch merging layer for patch fusion and downsampling (Section 3.3.1). Notably, recognizing the significance of retaining all potentially relevant target features comprehensively to furnish the decoder with ample target feature information, we preserve the output of each Transformer encoder layer and forward them into subsequent Multi-layer Feature Aggregation (MFA) modules through shortcuts (Section 3.3.3).

In the decoder, we SPP-DCN module to receive feature information from the MFA module. This module dynamically adjusts the perception range, enabling comprehensive attention to incorporate local fine-grained details across different scales. Additionally, we employ ASFF module to adaptively fuse the feature maps originating from the first three layers of the Transformer encoder, which maximizes the exploitation of low-level feature information in the encoder (Section 3.4.2).

3.3. Encoder Based on Effective Transformer

The Transformer has significantly advanced the evolution of deep network models in various fields and tasks. Presently, a majority of large models are derived from the evolution of the Transformer. Likewise, we inherit the advantages of self-attention and employ efficient Transformers in the encoding stage.

Overall, the Transformer backbone we use can be divided into four stages (Figure 6a). Each stage includes a patch merging module and multiple stacked Transformer encoders. The patch merging module is responsible for image patch merging and downsampling, while the Transformer encoder module computes self-attention to obtain global contextual information.

3.3.1. Overlap Patch Merging

ViT divides the input image into patches using fixed divided windows, which poses the issue of no overlap information exchange between windows. Conversely, Swin Transformer utilizes sliding windows to carry contextual window information. Unlike both approaches, we ingeniously utilize convolution to construct patches with overlap relationships. This approach simplified patch construction between overlapping windows while ensuring efficient contextual correlation with local information, as shown in Figure 6b. Specifically, we use stride and kernel size to control the overlap range and also map the patch embedding dimension at the same time.

We use $OS$ to represent the downsampling scale. Taking a stage i ($i=1,2,3,4$) as an example, we use convolutional kernels with a stride $s_i$ ($s_i>1$) to establish patches with overlapping relationships on the input feature map ${\displaystyle \frac{H}{O{S}_{i-1}}}\times {\displaystyle \frac{W}{O{S}_{i-1}}}\times E{D}_{i-1}$. We design the kernel size $k_i$ and padding $p_i$ to downsample the feature map. Therefore, the size of the output feature map becomes ${\displaystyle \frac{H}{O{S}_i}}\times {\displaystyle \frac{W}{O{S}_i}}\times E{D}_i$. The specific parameters for each stage are elaborated on in

Table 1. Parameters of the Transformer backbone in the encoder. We got inspiration from SwinTransformer [41] and ResNet [47].

Stage i	Overlap Patch Merging			Efficient Multi-Head Attention
	Overlap Convolution (k, s, p)	Embedding Dimension (ED)	Output Stride (OS)	Multi-Head Number	Depth	Reduction Ratio
Stage 1	(7, 4, 3)	64	4	1	3	8
Stage 2	(3, 2, 1)	128	8	2	4	4
Stage 3	(3, 2, 1)	320	16	5	6	2
Stage 4	(3, 2, 1)	512	32	8	3	1

. It is worth noting that we use larger convolutional kernels in the first stage to achieve greater downsampling, thereby reducing the computational costs of the backbone.

3.3.2. Effective Transformer Encoder

The Transformer encoder is tasked with establishing contextual relationships among input patch embeddings. A fully complete Transformer encoder includes the following components: layer normalization, self-attention, feedforward neural network (FNN), and two residual connections, as shown in Figure 7.

Effective Multi-Head Attention. Due to the quadratic computational complexity of self-attention, it poses a significant computational burden, particularly with larger input image sizes. Inspired by the Pyramid Vision Transformer (PVT) [43], we introduce SRA to alleviate computational costs. Given the matrices $K,V$, we reduce their dimensions through $Reshape$ and $Linear$ transformations, represented as follows:

$$\begin{array}{cc}\hfill K^{\prime },V^{\prime }& =Reshape\left({\displaystyle \frac{N}{R}},D·R\right)\left(K,V\right)\hfill \\ \hfill \widehat{K},\widehat{V}& =Linear\left(D·R,D\right)\left(K^{\prime },V^{\prime }\right)\hfill \end{array}$$

(5)

where R denotes the reduction ratio. Notably, reduction solely applied to $K,V$ will not change the output dimension of the self-attention, and it decreases the computational complexity of the self-attention from $\mathcal{O}\left(N^{2}D\right)$ to $\mathcal{O}\left({\displaystyle \frac{N^{2}D}{R}}\right)$.

Moreover, to comprehensively capture spatial information with varying emphases and bolster the backbone’s generalization, we employ multiple mapping matrices to transform the input sequence into $Q,K,V\in {\mathbb{R}}^{N\times D_h}$ located in distinct spatial domains, and then compute multi-head self-attention parallelly using Equation (1). Subsequently, we combine the results of multi-head self-attention through mapping, which fuses the contextual relationships from multiple spatial domains, represented as follows:

$$DPMSA\left(s\right)=[DPS{A}_1\left(s\right);DPS{A}_2\left(s\right);\cdots ;DPS{A}_h\left(s\right)]·W$$

(6)

where $W\in {\mathbb{R}}^{h·D_h\times D}$ is the mapping matrix, and h is the multi-head number. The effective multi-head attention is shown in Figure 7.

FNN module. The feedforward neural network plays a pivotal role in introducing non-linearity to the outputs post self-attention. Given the absence of positional encoding in the Transformer encoder, it can be particularly important to compensate for positional information in FNN. Specifically, we employ an added $3\times 3$ depthwise convolution with zero padding in FNN, which incorporates positional information efficiently [48]. The FNN module could be represented as follows:

$$FNN\left(s\right)=MLP\left(GeLU\left(DWCon{v}_{3\times 3}\left(MLP\left(s\right)\right)\right)\right)$$

(7)

Overall, the Transformer encoder can be represented as follows:

$$\begin{array}{cc}\hfill s_l^{\prime }& =DPMSA\left(LN\left(s_{l-1}\right)\right)+s_{l-1}\hfill \\ \hfill s_l& =FNN\left(LN\left(s_l^{\prime }\right)\right)+s_l^{\prime }\hfill \end{array}$$

(8)

where $s_l$ represents the sequence of the l-th layer of the Transformer encoder, and $LN$ represents layer normalization.

3.3.3. MFA Module for Feature Fusion

We employed a multi-hierarchical feature extraction architecture based on the Transformer, where low-level features emphasize fine-grained relationships, while high-level features capture global contextual relationships. To fully utilize both global and local multiscale information, we deploy an MFA module to aggregate results from all Transformer stages, as shown in Figure 5. Specifically, we first map all feature maps through an MLP layer to a unified channel dimension to balance the contribution level of each stage. Subsequently, we use the feature map with an $OS$ of 16 as the reference and align and unify the other feature maps accordingly. This process involves bilinear upsampling the feature map with an $OS$ of 32, and downsampling the feature maps with an $OS$ of 4 and 16. Finally, we merge these four feature maps through concatenation. This module effectively balances efficiency and memory consumption. The details of the structure are shown is Figure 5c.

3.4. Decoder Based on Adaptive Aggregation

In the decoding stage, we aim to focus attention on the target areas by fully utilizing encoding feature information, refining target boundaries, and sharpening segmentation results. To comprehensively correlate the multiscale range in the defect areas, we propose the SPP-DCN module. Additionally, to effectively reuse the information extracted by the encoder, we employ the ASFF module.

3.4.1. Serial0-Parallel Pyramid Based on Atrous Deformable Convolution Module

Inspired by DeepLabv3 [49] and DenseASPP [50], we have designed a multi-branch pyramid structure with a Deformable Convolution module called SPP-DCN, as shown in Figure 5e. This module connects the MFA module from encoder and dynamically adjusts the perception field of defects efficiently.

In our approach, we use 7 parallel branches in the pyramid. Among them, two are designed to retain the essential original feature information, and one branch initializes the receptive field using a $1\times 1$ Depthwise Separable Convolution. Furthermore, we employ another branch with a global average pooling layer to maximize the receptive field. The remaining 5 branches utilize Deformable Convolutions with varying dilation rates. Inspired by DenseASPP [50], we set the dilation rates to $\{3,6,12,18,24\}$. It is worth noting that in addition to parallel branches, we also incorporate a serial structure. Specifically, the outputs of branches with Deformable Convolution are stacked with the input of the subsequent branch through concatenation operation. This strategy is aimed to establish cascading dilated convolution relationships, compensating for the loss of local perceptual details with larger dilation rates.

In this module, another improvement is the redesign of the Deformable Convolution module with dilation rates, as depicted in Figure 5a. Firstly, we followed the method proposed by Huang et al. [51], which involves incorporating modulation scalar in Deformable Convolutions to distinguish whether the areas we offset are regions of interest. Secondly, we introduce group convolution in Deformable Convolutions. Specifically, each group has its own sampling offsets, enabling different groups within a convolutional layer to exhibit distinct spatial aggregation patterns, which enriches features for decoding. Therefore, the Deformable Convolutions (Equation (3)) can be enhanced as follows:

$$y\left(p_o\right)=\sum _{g\in G}\sum _{p_n\in R}{m}_{gk}·w_g\left(p_n\right)·x_g\left(p_o+p_n+\Delta p_n\right)$$

(9)

where G is the total number of groups. $m_{gk}$ represents the modulation scalar of the k-th sampling point in the g-th group, and it undergoes softmax normalization along the k dimension. w represents the convolutional learning weights along the g-th group. Both $m_{gk}$ and $\Delta p_n$ are learned from the input feature map. We introduce dilation rates in Deformable Convolutions to allow $p_n$ to span larger regions while $\Delta p_n$ achieves minor adjustments, allowing the convolution to to dynamically focus more effectively on distant feature relationships.

Subsequently, we employ channel-wise feature weighting using the SE module, followed by channel mapping through an MLP module, as shown in Figure 8. Furthermore, we integrate a shortcut path to preserve the original features, designed to prevent over-fitting.

3.4.2. Adaptive Spatial Feature Fusion Module

To fully capitalize on the defect feature information extracted by the encoder, we establish shortcut connections from the first three stages between the encoder and decoder. These features are particularly beneficial for locating defect areas and refining edge details. The ASFF module is introduced to receive feature maps that have been aligned in channels and scales through separate $1\times 1$ convolutions and upsampling, as shown in Figure 5.

In the ASFF module, we uniformly compress and concatenate the three input features, then obtain the weight parameters for each feature through convolution and softmax. Finally, we perform weighted summation on the original input features, which can be represented as follows:

$$\begin{array}{cc}\hfill W_1,W_2,W_3& =softmax(f_{64\to 1}^1,f_{128\to 1}^2,f_{320\to 1}^3)\hfill \\ \hfill ASFF(f^1,f^2,f^3)& =W_1·f^{1\to 1}+W_2·f^{2\to 1}+W_3·f^{3\to 1}\hfill \end{array}$$

(10)

where $f^i$ represents the feature from the i-th stage, $f_{·\to ·}$ represents the feature dimension transformation, and $f^{·\to ·}$ represents the feature scale alignment. As described, we dynamically adjust the contributions of the feature maps through the ASFF module, adapting the low-level information required for the decoding stage.

3.5. Loss Function

We train the segmentation network by minimizing a multi-loss function:

$$L_{train}=L_{last\_out}+\gamma L_{aux\_out}$$

(11)

where $L_{last\_out}$ represents the final output of the decoder, while $L_{aux\_out}$ represents the prediction loss comes from the FCN head, as shown in Figure 5. $\gamma$ represents the balancing weight, which was set at 0.5 in our experiment (the experiment is shown in Section 5.5).

The FCN head is exclusively utilized during training, with its specific structure outlined in Figure 5a. Our purpose is to establish a shortcut to directly supervise the training of the backbone, accelerating the convergence speed of the network.

For each loss, specifically, we supervise them with the linear combination of Cross-Entropy (CE) loss and Dice loss [52], aiming to balance the contributions between positive and negative samples. The composite loss function is formulated as follows:

$$L={\alpha L}_{CE}+\beta L_{Dice}$$

(12)

where $\alpha$ and $\beta$ are balancing weights. The CE loss is defined as follows:

$$L_{CE}=-{\displaystyle \frac{1}{n}}\sum _{i=1}^n{y}_i·ln{p}_i+(1-y_i)ln(1-p_i)$$

(13)

where n represents the number of pixels in the image, $p_i$ denotes the predicted classes of the pixel, and $y_i$ denotes the label classes of the pixel. The Dice loss is introduced to balance the training loss where background learning dominates in the CE loss:

$$L_{Dice}=1-{\displaystyle \frac{{\sum }_{i=1}^n{p}_i·y_i+\epsilon }{{\sum }_{i=1}^n{p}_i+y_i+\epsilon }}-{\displaystyle \frac{{\sum }_{i=1}^n(1-p_i)(1-y_i)+\epsilon }{{\sum }_{i=1}^{n}2-p_i-y_i+\epsilon }}$$

(14)

where $\epsilon$ is the smooth parameter introduced to prevent overly large changes. It was set to 1 in our experiment.

4. Experiment Results

4.1. Datasets

The RGB image dataset used in the experiments was collected by a cable climbing robot, with detected bridge locations predominantly situated at some large cable-stayed bridges in Shanxi Province, China. We manually conducted a rough screening of images with defects as the dataset and uniformly cropped them to a size of $960\times 540$. The dataset comprises a total of 2304 images, which were divided into training, validation, and test sets according to a ratio of 7:2:1. Subsequently, we manually annotated the ground truth of the defects for supervising network training, which included but was not limited to circumferential cracks, longitudinal cracks, scratches, spalling, and attachments. To ensure the robustness of our network in complex backgrounds, we intentionally retained elements such as the bridge-climbing robot itself and natural environmental factors, while maintaining an oblique perspective during image capture, as shown in Figure 9.

To avoid over-fitting, it is crucial to mention that before training the model, we randomly cropped images patch to a size of $512\times 512$, and performed common data augmentations to enhance the diversity of input shapes, including but not limited to Random Horizontal Flip, Random Scale Crop, Random Gaussian Blur, and Random Rotate.

4.2. Evaluation Metrics

To report the performance of bridge cable defect segmentation, we employed the following evaluation metrics: accuracy (Acc), mean Intersection over Union (mIoU), and F1 score.

• Acc indicates the agreement between the predicted class and the ground truth labels for all pixels, and it can be expressed as follows:

  $$\mathrm{Acc}={\displaystyle \frac{N_{TP}+N_{TN}}{N_{TP}+N_{FP}+N_{FN}+N_{TN}}}$$

  (15)

  where $N_{TP}$ is the number of true positives, $N_{FP}$ is the number of false positives, $N_{FN}$ is the number of false negatives, and $N_{TN}$ is the number of true negative.
• mIoU indicates the mean Intersection over Union (IoU) between the predicted class and the ground truth label, which can be expressed as follows:

  $$\mathrm{mIoU}={\displaystyle \frac{1}{n+1}}\sum _{i=0}^n{\displaystyle \frac{N_{TP}}{N_{TP}+N_{FP}+N_{FN}}}$$

  (16)

  where n represents the number of classes.
• F1 score is a harmonic mean of precision and recall, and it can be calculated as follows:

  $$\mathrm{F}1={\displaystyle \frac{2\times Precision\times Recall}{Precision\times Recall}}$$

  (17)

  where

  $$\begin{array}{cc}\hfill \mathrm{Precision}& ={\displaystyle \frac{N_{TP}}{N_{TP}+N_{FP}}}\hfill \\ \hfill \mathrm{Recall}& ={\displaystyle \frac{N_{TP}}{N_{TP}+N_{FN}}}\hfill \end{array}$$

  (18)

The metrics mentioned range from 0 to 1, where values closer to 1 indicate higher segmentation accuracy.

4.3. Implementation Details

We implemented our methods on Pytorch 1.8.0 and conducted training on a workstation equipped with four Nvidia RTX 3090 GPUs. SyncBatchNorm was applied before each weight layer in our implementation for parallel training across multiple GPUs. For training, we determined the optimal optimizer parameters through cross-validation. The specific experimental results are presented in Appendix A. The best parameter settings identified were to use the SGD optimizer with an initial momentum of 0.9 and a weight decay of $5\times {10}^{-4}$. The learning rate was scheduled using a polynomial exponential decay, starting with an initial value of 0.007.

We pre-trained the backbone on the ImageNet-1K dataset and randomly initialize the remaining structure. During fine-tuning with the bridge cable defect dataset, we trained the models for 100 epochs with a batch size of 15. Subsequently, we selected the model with the best mIoU for testing.

4.4. Comparison with Regard to State-of-the-Art Models

In this section, we evaluate the segmentation performance of Trans-DCN in detecting bridge cable defects through a series of experiments. We compare Trans-DCN with various baselines, such as DeepLabv3+, DANet, SegNeXt, ViT-CoMer, and others, employing different backbones. We used cross-validation experiments to determine the optimal training parameters, ensuring that these models achieve the best performance on the bridge cable defect dataset we established. The optimal training parameters identified are detailed in Appendix B. Those segmentation results are shown in

Table 2. Comparison with state-of-the-art methods. ↑ represents a better segmentation performance, ↓ represents lower computing requirements.

Method	Model Architecture	Params. (M) ↓	GFlops ↓	Evaluation ↑
				Acc (%)	mIoU (%)	F1-Score (%)
Convolution- based	U-Net [53]	13.39	124.49	96.33	78.21	74.25
	FCN (VGG-16) [33]	35.31	148.53	96.90	79.12	76.31
	SegNet (VGG-16) [54]	29.44	160.68	97.34	81.41	79.25
	PSPNet (ResNet50) [55]	46.71	59.21	97.03	78.26	74.45
	DeepLabv3 (ResNet50) [49]	41.99	173.79	97.12	78.69	74.98
	DeepLabv3+ (MobileNet-V2) [56]	41.83	33.75	97.02	78.90	76.17
	DeepLabv3+ (ResNet101) [56]	74.87	82.88	97.09	80.72	78.32
	DenseASPP (DenseNet121) [50]	9.20	43.19	96.79	75.67	69.33
	DANet (ResNet101) [57]	66.56	283.44	96.86	75.61	69.06
	LR-ASPP (MobileNet-V3) [58]	30.22	20.07	96.57	76.93	72.99
	SegNeXt (Base) [59]	29.91	41.55	94.66	65.34	56.13
Transformer- based	SegFormer (B2) [60]	47.22	71.36	97.31	81.47	79.36
	Swin-Unet (Tiny) [61]	27.17	5.92	96.71	78.02	74.61
	Bi-Former (Base) [62]	56.81	91.10	95.12	68.02	58.25
	ViT-CoMer (Small) [63]	60.50	1194.16	96.70	77.32	73.49
	Trans-DCN (ours)	44.96	52.03	97.49	82.63	80.89

. It is worth noting that, along with evaluation metrics (including Acc, mIoU, and F1 score), we also provide the parameter count and computational cost of each model, which is because segmentation accuracy is influenced by these factors. Our efficient model aims to minimize the parameter count and computational cost while ensuring excellent evaluation. We selected some challenging cases for model prediction, and the visualization of the segmented defect area is presented in Figure 10.

As shown in Table 2, our model, employing with effective Transformer as the encoder and adaptive aggregation as the decoder, achieves a performance of 82.63% in mIoU, attaining state-of-the-art performance among the baseline models. Compared to the latest baseline (ViT-CoMer), our model demonstrates an accuracy improvement of 5.31%. Furthermore, compared to the state-of-the-art baseline (SegFormer), our accuracy improved by 1.16%. Our model exhibits a significant improvement and outperforms other baselines in defect extraction by dynamically adjusting the perceived field and promptly adjusting the utilization of backbone features. Notably, our model’s parameter count and computational cost remain lower, indicating that our proposed method is not only high-performing but also efficient. Specifically, our parameter count and computational cost are almost as same as the PSPNet baseline, but our accuracy has been improved by 4.37% (mIoU). Moreover, compared with ViT-CoMer using the UperNet method, our proposed method achieves a parameter reduction of 15.34 M and a computation reduction of 1096 GFlops. This highlights the significant efficient improvement achieved by using an effective Transformer and Depthwise Separable Convolution fully.

Benefiting from the powerful Transformer serving as the backbone for the full contextual feature correlation mechanism, our method excels in detecting large-scale defect distributions, as shown in Figure 10b,c,g,k. Our method achieved defect predictions closest to the ground truth compared to other baselines. Taking Figure 10b,c as examples, our method has a comprehensive perception of defects in the image, effectively capturing even the smallest local damages compare to convolutional models represented by SegNet. DenseASPP and DANet also display shortcomings in predicting long cracks, resulting in fractured segmentation areas. Transformer models represented by Bi-Former tend to segment areas exceeding the boundaries, while ViT-CoMer shows incorrect segmentation. Additionally, in Figure 10j, SegNet, LR-ASPP, and Bi-Former show more false positives, mainly due to the presence of interfering textures outside the circumferential crack. However, our method, by fully considering multi-level features, was able to suppress this noise and preserve the integrity of the target.

Additionally, thanks to the well-designed adaptive Deformable Convolution and dynamic spatial aggregation module, our method accurately predicts strips defects and maintains complete edges, as shown in Figure 10a,d,h,m. Models based on fixed convolution patterns inevitably exhibit discontinuities when faced with elongated features, as seen in PSPNet and DANet (particularly highlighted in Figure 10d,h,m). Models based on ASPP, such as LR-ASPP and DenseASPP, encounter difficulties in aggregating sufficient spatial information when dealing with this type of challenge, as atrous convolutions lose their advantage in strip targets. Although our approach also involves atrous convolutions, it promptly corrects unnecessary expansions through a combination with Deformable Convolutions. Transformer-based models, such as Bi-Former and ViT-CoMer, still perform disappointingly in this scenario. The SegFormer baseline performs similarly to ours, but there is a significant difference observed in Figure 10h.

Meanwhile, we intentionally select defect images featuring shallow scratches or captured in environments with low or uneven lighting conditions to illustrate the robustness of our method under challenging conditions, as shown in Figure 10e,i. The results demonstrate that our model is capable of fully exploring potential defect information, with sharp edges observed. However, some false positives are present simultaneously. The prediction is incomplete in Figure 10e, while SegFormer performs satisfactorily in Figure 10i by exploring complete information at multiple granularities.

In most cases in Figure 10, our detection accurately captures the area of defects adequately. Specifically, Figure 10a,d,h,j,l highlight the strong segmentation integrity of our model. Therefore, our proposed network sets a new state-of-art performance in bridge defect segmentation.

5. Analysis

5.1. Ablation Study

In this section, we introduce ablation experiments, wherein MFA (Section 3.3.3), SPP-DCN (Section 3.4.1), ASFF (Section 3.4.2), and Aux-Loss (Section 3.5) are included as ablation units. The results of these experiments are shown in

Table 3. Ablation study. ↑ represents a better segmentation performance.

Model	MFA	SPP-DCN	ASFF	Aux-Loss	Evaluation ↑
					Acc (%)	mIoU (%)	F1-Score (%)
1	-	-	-	-	95.55	77.12	70.23
2	✓	-	-	-	97.13	80.29	77.89
3	✓	-	-	✓	97.14	80.48	78.03
4	✓	✓	-	✓	97.35	81.42	79.24
5	✓	-	✓	✓	97.30	81.14	78.87
6	✓	✓	✓	✓	97.49	82.63	80.89

. Specifically, in the model without using the MFA module, only the output of the last stage is utilized in the decoder. In the model without the SPP-DCN module, a regular convolution is directly employed for the connection. In the model without the ASFF module, the shortcut is directly removed.

From Table 3, it is evident that the accuracy of our model can be increased by up to 5.5% (mIoU) with our improvements. Specifically, solely adding the MFA module resulted in a 3.1% increase in mIoU, indicating the necessity of fully integrating multiscale feature maps in the encoder. Subsequently, incorporating auxiliary training loss led to a slight improvement in mIoU during validation. Furthermore, the cross-validation model (4–5), which integrates the SPP-DCN module or ASFF module, exhibited a 1% or 0.7% increase in mIoU, respectively, highlighting the effectiveness of each proposed module in the model. Additionally, captured from Figure 11, it can be observed that removing each effective module causes a decrease in the validation accuracy of our model. Overall, each well-designed module in our model has proven to be indispensable, resulting in a final mIoU score of 82.63%.

5.2. Influence of Transformer Backbone

In this section, we compare and evaluate the segmentation accuracy and parameter count using different backbones to demonstrate the performance of our proposed method. We selected several convolution-based and Transformer-based backbones, using their default parameters from the officially provided codes and enabling pre-training. It is worth noting that for backbones with multiple versions, we attempted to match them with parameter counts comparable to our proposed method. Their version and the experimental results are shown in

Table 4. Comparison with different backbones in the encoder. ↑ represents a better segmentation performance, ↓ represents lower computing requirements.

Method	Backbone	Params. (M) ↓	Evaluation ↑
			Acc (%)	mIoU (%)	F1-Score (%)
Convolution- based	ResNet-101 [47]	42.5	96.37	78.70	73.69
	ResNeXt-101 (32 × 8d) [64]	86.74	96.60	79.93	76.72
	Xception [56]	37.87	89.32	57.16	48.66
	MobileNet-V2 [65]	15.4	96.96	75.14	66.91
	ConvNeXtV2-B [66]	88.72	93.37	65.35	55.27
Transformer- based	Swin Transformer-T [41]	96.52	93.62	76.76	72.81
	Swin TransformerV2-T [67]	27.58	95.94	69.00	59.86
	ViT-CoMer-S [63]	37.34	96.90	78.98	75.94
	Proposed method	24.2	97.49	82.63	80.89

.

Our proposed method, compared to the latest Transformer-based backbone ViT-CoMer, achieves an improvement of 3.6% in mIoU, with a parameter reduction of 13 M. Compared to the state-of-the-art convolution-based backbone ResNeXt-101, our method still achieves a 2.6% mIoU improvement, with a parameter reduction of 62.54 M, fully highlighting the advantages of our method. Additionally, our method outperforms the backbone MobileNet-V2 by 7.5%, with an increase of only 8.8 M parameters. Therefore, our method stands out as the most cost-effective choice.

Additionally, we conducted experiments on the embedding dimensions in the backbone. We compared the segmentation accuracy of the original embedding dimensions in ResNet [47] with our optimized dimensions. The experimental data are presented in

Table 5. Accuracy, parameter count and computational cost as a function of the embedding dimension of the Transformer backbone. ↑ represents a better segmentation performance, ↓ represents lower computing requirements.

Stage i	Embedding Dimension	Multi-Head Number	Params. (M) ↓	GFlops ↓	Evaluation ↑
					Acc (%)	mIoU (%)	F1-Score (%)
1	64	1	92.06	69.72	97.41	81.89	79.82
2	256	4
3	512	8
4	1024	16
1	64	1	24.2	39.44	97.49	82.63	80.89
2	128	2
3	320	5
4	512	8

. It is noteworthy that the multi-head number in the multi-head attention mechanism varies with the embedding dimensions. We ensured that every 64 dimensions correspond to one head in the self-attention calculation, thereby guaranteeing richer spatial learning patterns.

As shown in the Table 5, our optimized embedding dimensions not only reduce the parameter count by 73% but also improve segmentation accuracy, which indicates that the dimensions we designed basically meet the efficient requirements for cable defect segmentation.

5.3. Influence of SPP-DCN

In this section, we individually evaluate the effectiveness of the internal structure of the SPP-DCN module, as shown in Figure 5e. We conducted an ablation study specifically designed for this module, wherein the shortcut, MLP, and SE modules serve as ablation units. The experimental results are presented in

Table 6. Ablation study based on SPP-DCN. ↑ represents a better segmentation performance.

Model	Shortcut	MLP	SE	Evaluation ↑
				Acc (%)	mIoU (%)	F1-Score (%)
1	-	-	-	97.29	81.03	80.25
2	-	✓	✓	97.50	82.47	80.65
3	✓	-	-	97.38	80.10	80.22
4	✓	-	✓	97.46	81.14	80.21
5	✓	✓	-	97.37	82.16	80.31
6	✓	✓	✓	97.49	82.63	80.89

.

From the table, it is notable that simply adding a shortcut after the atrous Deformable Convolution (model 3) does not yield a positive effect (mIoU decreases by 1%). This is because the features do not pass through subsequent fully connected and channel-attention layers, rendering the residual connection ineffective. Conversely, adding both MLP and SE modules after the convolution simultaneously (model 2) can increase the mIoU by 1.5%, which demonstrates that adding both modules can effectively enhance the adaptively perceived defect information. Simultaneously, the cross-validation (model 4–5), which integrates the MLP module or SE module, exhibited a similar 1.1% increase in mIoU. Consequently, the conclusion can be drawn that SPP-DCN’s capabilities can be fully enhanced by combining all three methods (model 6), producing the best segmentation outcomes (1.6% improvement).

Moreover, we provide the visualization of the receptive fields compared to the ASPP module, as shown in Figure 12. Our receptive fields in SPP-DCN are more uniform and flexible, which is advantageous for detecting scattered defects. The examples on the right side well demonstrate that our design can better adjust the attention area, effectively reducing the possibility of mis-segmentation.

In addition, we conducted an experiment with branches with different dilation rates, as shown in

Table 7. Accuracy as a function of the number of branches in the SPP-DCN. RF represents receptive field. ↑ represents a better segmentation performance.

Branches	Max RF	Evaluation ↑
		Acc (%)	mIoU (%)	F1-Score (%)
(6, 12, 18)	73	96.29	75.95	71.62
(3, 6, 12, 18)	79	96.44	78.21	75.01
(6, 12, 18, 24)	122	96.76	79.94	77.25
(3, 6, 12, 18, 24)	128	97.49	82.63	80.89

. The experimental results indicate that setting the dilation rates to $\{3,6,12,18,24\}$ achieved the highest segmentation mIoU, which is 6.7% higher than the dilation rates of $\{6,12,18\}$. Our branch setup provides a maximum receptive field of 128, enabling better perception of defects over a large scale while ensuring the detection of small-scale features. Consequently, this design is more effective in handling the varying feature of cable defects.

5.4. Influence of ASFF

In this section, we analyze the impact of the channel alignment scale within the ASFF on the performance. As illustrated in the network architecture (Figure 5), we fully exploit the low-level features from the encoder through shortcuts and perform adaptive aggregation in the decoder. Before softmax calculation, it is crucial to carefully determine the channel alignment of each feature map, as it influences contributions post-concatenation with the subsequent SPP-DCN module. We directly consider the channel dimension of the feature maps in the three shortcuts, and the experimental results are detailed in

Table 8. Accuracy, parameter count and computational cost as a function of the channel alignment dimension of the ASFF. ↑ represents a better segmentation performance, ↓ represents lower computing requirements.

Model	Channel	Params. (M) ↓	GFlops ↓	Evaluation ↑
				Acc (%)	mIoU (%)	F1-Score (%)
1	64	0.06	0.7	97.47	82.12	80.16
2	128	0.2	2.64	97.49	82.63	80.89
3	320	0.99	15.43	97.33	81.80	79.81

.

In Table 8, we show performance alongside parameter count and computational cost, varying with channel dimension. Notably, setting $C=128$ yields commendable performance while maintaining competitive computational cost. As the dimension increases, there is a decline in mIoU, coupled with a substantial rise in computational cost. Conversely, reducing the dimension compromises performance, rendering it unsatisfactory.

5.5. Influence of Composite Loss Function

In this section, we conducted experiments to evaluate the weight of the composite loss function during model training (Section 3.5). We focused on understanding the impact of the weight ratio between the CE loss $\alpha$ and the Dice loss $\beta$ on both training accuracy and convergence speed. Notably, we employed a step size of 0.5 for the CE loss and 0.25 for the Dice loss, as larger Dice loss values may induce training instability. Additionally, we also included $\gamma$ as a variable in the experiment. The experimental results are detailed in the

Table 9. Accuracy as a function of difference ratio of composite loss functions. ↑ represents a better segmentation performance.

Ratio			Evaluation ↑
$\mathit{\alpha }$	$\mathit{\beta }$	$\mathit{\gamma }$	Acc (%)	mIoU (%)	F1-Score (%)
1	0	0.5	97.28	81.40	79.27
1	0.5	0.5	97.37	81.71	79.65
1.5	0.5	0.5	97.39	82.09	80.19
2	0.5	0.5	97.49	82.63	80.89
2.5	0.5	0.5	97.42	82.41	80.36
3	0.5	0.5	97.37	81.95	80.01
2	0.25	0.5	97.36	81.94	80.00
2	0.75	0.5	97.44	82.29	80.44
2	0.5	0	97.44	82.06	80.10
2	0.5	0.25	97.44	82.17	80.27
2	0.5	0.75	97.55	82.46	80.60
2	0.5	1	97.46	82.20	80.30

.

In Table 9, we can observe that the composite loss function with $\alpha =2$ and $\beta =0.5$ achieves the highest performance with 82.63% in mIoU (1.23% improvement). Notably, altering the values of $\alpha$ and $\beta$ results in a decline in the validation segmentation performance, and altering the $\gamma$ will have a very slightly impact on the evaluation. Furthermore, Figure 13 illustrates that the composite loss function with $\alpha =2$ and $\beta =0.5$ exhibits superior convergence and optimization capabilities compared to other combinations.

6. Discussion

Our work serves as foundational research and opens up various possibilities for future endeavors in the field of cable defect detection such as precise defect localization and measurement. However, it is important to acknowledge the limitations of our work. For instance, the presence of extremely small cracks lacking proper annotation may compromise the network’s detection capabilities in this regard. Additionally, imbalance in bridge cable defect categories within our dataset poses a challenge for multi-class semantic segmentation. To address these challenges, our future endeavors will concentrate on few-shot semantic segmentation models capable of efficiently detecting relevant defects with minimal reliance on a large number of samples for specific classes. Furthermore, we are committed to expanding our dataset to encompass a broader range of bridge cable defect scenarios, thereby providing a robust benchmark for training and evaluating our models. We are interested in determining whether the proposed network can maintain its excellent extraction capabilities when applied to defect domains beyond bridge cables. In future work, we will evaluate our method with more diverse data resources to test its robustness. Furthermore, we will investigate the network’s adaptability to other data input sources (e.g., infrared sensors), exploring its versatility and potential for broader applications.

7. Conclusions

This paper presents an efficient and practical bridge cable defect segmentation network, Trans-DCN, which fully integrates Transformers and convolutions, achieving state-of-the-art defect segmentation performance within the encoder–decoder framework. The proposed model employs an efficient Transformer as the backbone, effectively leveraging features extracted by self-attention, which contains both local fine-grained and global contextual information. Furthermore, multiple-layer features are judiciously reused in the encoder. In the decoder, SPP-DCN is designed to dynamically adjust the perception range, concentrating the attention based on the distribution characteristics of the defects. Additionally, the model incorporated several shortcuts between the encoder and decoder, and adaptive fusion of multi-layer features was used in the decoder. Moreover, the introduction of Depthwise Separable Convolutions and enhancements to the loss function serve to improve the efficiency and convergence speed of the model.

For the detection of bridge cable defects, we collected actual captured images dataset for conducting experiments. The experimental results emphasize that in comparison to DeeplabV3+, SegFormer, Swin-Unet, and other baseline models, our model achieved state-of-the-art performance in detecting bridge cable defects. Specifically, our model achieves an Acc of 97.49%, an mIoU of 82.63%, and an F1 score of 85%. Simultaneously, we conducted multiple experiments to demonstrate the effectiveness of each module within our model.

Our designed network imposes no specific requirements on the characteristics of the data, needing only RGB images, and no preprocessing is required to remove irrelevant background parts. This flexibility means the data source can be from cable-climbing robots, UAVs, or manual collection. Our proposed method is well suited for offline, fast, and automated analysis of bridge cable surface defects, providing a cost-effective solution.