A Novel Technique for High-Efficiency Characterization of Complex Cracks with Visual Artifacts

Abstract

In this paper, we introduce SHSnet, an advanced deep learning model designed for the efficient end-to-end segmentation of complex cracks, including thin, tortuous, and densely distributed ones. SHSnet features a non-uniform attention mechanism, a large receptive field, and boundary refinement to enhance segmentation performance while maintaining computational efficiency. To further optimize the model’s learning capability with highly imbalanced datasets, we employ a loss function (LP) based on the focal Tversky function. SHSnet shows very high performance, with values of 0.85, 0.83, 0.81, and 0.84 for precision, recall, intersection over union (IOU), and F-score, respectively. It achieves this with 10× fewer parameters than other models in the literature. Complementing SHSnet, we also present the post-processing unit (PPU), which analyzes crack morphological parameters through fracture mechanics and geometric properties. The PPU generates scanning lines to accurately compute these parameters, ensuring reliable results. The PPU shows a relative error of 0.4%, 1.2%, and 5.6% for crack number, length, and width, respectively. The methodology was benchmarked on complex ECC crack datasets as well as on multiple online datasets. In both of these cases, our results confirm that SHSnet consistently delivers superior performance and efficiency across various scenarios as compared to the methods in the literature.

1. Introduction

Over the years, many technologies have been developed to characterize damages in concrete infrastructures. These can be classified based on the sensing principle utilized: stress waves [1,2], electrical properties [3,4], optical fibers [5,6], or digital image correlation (DIC) [7]. Among these techniques, the vision-based technique is considered apt because of its simple implementation as well as direct and easy result interpretation.

The literature on vision-based systems of crack characterization is vast and ever-growing. A more detailed review can be found in reference [8]. Generally speaking, studies such as [9,10,11] have reinforced that deep learning models based on convolutional neural networks (CNNs) outperform machine learning models, feed-forward neural networks (FNNs), and conventional image processing techniques. Recently, many techniques based on deep learning methodologies have been proposed [12]. They have shown stellar performance in many different scenarios on normal concrete cracks. These models can be classified based on architecture: (A) mask R-CNNs [13], which require pixel-level labels and bounding boxes as ground truth, leading to incompatibility with this dataset and others; (B) convolutional autoencoders (such as [14,15]), which need to be trained on a large sample size of non-textured backgrounds and compute pixel-wise defects as the difference between the reconstructed image and the original image, working well with large defects on non-textured backgrounds [15] but not suitable for this work; and (C) encoder–decoder architectures, the most common for developing segmentation models for normal cracks [12], with different variations as discussed below.

Researchers working on normal concrete cracks collected from various parts of the world (for example, Silva et al. [16] in Denmark, Słonski [17] in the USA, Dung et al. [18] in Turkey, Yang et al. [19] in China, and Yeoh [20] in Singapore) found that VGG16-based models work well for the identification and segmentation of RC cracks. The FPHBN [21] model is a variation of VGG16-based models, where features at different scales are pooled to create a segmentation map. Other notable models include DeepCrack [22], CrackForest [23], and CrackTree [24].

SDDnet [25], loosely based on the DeepLabV3+ [26] architecture, is superior to these models and has a lower computational cost. Other computationally efficient crack segmentation models based on DeepLabV3+ include Sarmiento’s Model [27], which uses ResNet18 for the segmentation of pavement cracks in the Philippines, and a similar model by Li et al. [28] for segmenting cracks in concrete dams. Xu et al. [29] created an encoder using depth-wise convolutions adapted from MobileNet [30], while Song et al. [31] used MobileNet as an encoder to improve parameter efficiency in the segmentation of normal cracks.

UNet is another popular model for semantic segmentation, specifically targeting medical images [32]. More recently, the use of attention for superior medical image segmentation has been demonstrated through Attention-UNet [33], which adds an attention module to UNet, and UNet++, which uses a nested design and further improves computational efficiency through pruning. The importance of attention to improving crack segmentation was also validated with CrackSegnet [34] and CrackNex [35].

Cracks significantly differ in shape; thus, Hang et al. [36] proposed CrackUNet by tailoring UNet for crack segmentation. This model is superior in segmenting cracks in reinforced concrete structures compared to many other deep learning networks. Although there have been recent attempts to reduce the parameter space of deep learning models, such as UNet++ [37], IDSnet [38], and MixCrackNet [39], it remains unknown whether they can deal with complex cracks rather than complex backgrounds, which is significantly different, as will be discussed later.

A detailed survey and the training requirements of these models are reported in

Table 1. Crack segmentation models and their training requirements.

Model Name	Based on	Publication Year	Hardware	ECC Cracks	Training Resolution
MixCrackNet	Multiscale Attention	2024	3090	No	512 × 512
Dynamic Semantic Segmentation	Encoder–Crossor–Decoder	2023	3060	No	256 × 256
Typical-RC-AI	VGG-16	2018	1080 Ti	No	224 × 224
DcsNet	Resnet18+ Pyramid Pooling	2023	Titan XP	No	640 × 544
DeepCrack		2019	Titan-X	No	544 × 384
CrackGauGAN	Generative AI	2024	3090 Ti	No	256 × 256
Ji’s Model	DeeplabV3+	2020	Quadro P4000	No	512 × 512
SDDnet	Encoder+Decoder+ASPP	2019	4xTitan X	No	1024 × 512
CrackNex	Resenet 101+ Fusion Module	2024	Titan RTX	No	400 × 400
STRnet		2022	Titan XP	No	1024 × 512
CSegNet	DeeplabV3+; Transformer	2024	3080	No	560 × 380
IDSnet		2022	Titan XP	No	640 × 480
FPHBN		2019	Titan X	No	480 × 320
Unet++	U-Net	2019	2080 Ti	No	512 × 512
Attention-Unet	U-Net	2019		No	48 × 48
AFFnet	ResNet	2019		No	224 × 224
CrackSegnet	SegNet	2021	1080 Ti	No	512 × 512
Segnet	Segnet		1060	No	256 × 256
CrackUnet	U-Net	2021	1080 Ti	No	256 × 256
FCN	FCN	2019	1070	No	227 × 227
ECC1	U-Net	2022	3090	Yes	512 × 512
ECC2	U-net	2023	3090 Ti	Yes	480 × 352

. The following conclusions were drawn:

A.

Most crack segmentation models are developed to detect wide cracks; there are rarely any models demonstrated for complex cracks.

B.

Even though there is a large volume of models published, they are not optimized for complex crack segmentation.

C.

Most of the models require modern hardware for training, which is not always available in various laboratories.

D.

Training resolutions are large enough to identify context and have become essential for crack segmentation.

It is known that cracks that are tortuous, have high density, are thin (micro), and have high contrast between these features (HCRF) occurring within the same crack segment are harder to characterize. A crack with one or more of these features is hereby referred to as a complex (micro)crack. This is confirmed by our previous work [9] as well as by other researchers [40,41] who found that due to additional complexity, networks developed for normal concrete cracks do not work or perform suboptimally for the characterization of complex cracks. Complex cracks occur in many infrastructures, including buildings, pavement, dams, bridges, and tunnels [42,43,44,45]. Unfortunately, due to additional complexity, the methods discussed above do not perform optimally in these situations. The development of techniques capable of the automatic characterization of thin, high-density, tortuous, HRCF microcracks (especially under loads) thus remains an open problem in the literature. Conventional wisdom suggests that deeper networks solve more complex problems. A huge deep learning model would certainly be able to solve the segmentation of complex cracks but would need large computing resources, not to mention large-scale labeled datasets that need years of development. This makes it challenging to incorporate these models in practical conditions such as in industrial/construction applications and makes it inaccessible to not well-equipped domain-specific research laboratories, i.e., impractical.

In this work, we present an alternative method that is not limited by the restrictions of conventional methodologies. More specifically, a new pipeline to characterize complex (micro)cracks is developed. Our contributions are as follows:

• A novel computational model is presented that has been specifically tailored for the high-efficiency pixel-wise segmentation of complex cracks and the characterization of these cracks in incomplete/inaccurate segmentation maps in highly imbalanced datasets.
• SHSnet is developed to achieve the segmentation of complex cracks. It consists of (A) a large receptive field that allows better learning of contextual features, (B) boundary refinement units to achieve better boundaries, (C) residual multiscale connections for ‘attention’ for long-range features and boost feature representation capability, and (D) a novel loss function (LF) to train the network efficiently on a highly imbalanced dataset with HCRF cracks. Our network design is novel and different from SOTA models (as discussed in the previous section) for crack segmentation.
• A novel post-processing unit (PPU) is developed for crack morphological parameter calculations. The PPU is based on fracture mechanics, and geometric properties are proposed. Based on this, scanning lines are used to compute crack parameter length and width and the number of complex cracks directly from inaccurate/incomplete segmentation maps.
• Novel benchmarking methodologies were proposed for (A) testing the network on a wide variety of realistic cases of different concrete surfaces and (B) automated testing for the efficiency of image processing units.

The results demonstrate that the proposed pipeline works well with variations in surface and crack features caused by constituents and surface preparation techniques encountered in concrete structures.

2. Proposed Method

2.1. Deep Learning Model

2.1.1. Network Component and Design

Our network SHSnet is a type of encoder–decoder network, and its overall architecture is shown in Figure 1. It consists of 4 units: (a) encoding units (ENCs), (b) decoding units (DECs), (c) large receptive fields (RFs) + boundary refinement units (BNs), and (d) weighed multiscale ‘attention’ (WMA). The design philosophy and working mechanisms of each of these units are discussed as follows.

ENCs: It should be noted that the richness (amount) of contextual information is controlled by the depth and scale of the encoder requiring large computational resources. Our goal is to develop a network design that not only could deal with complex cracks but also could not be computationally expensive. Encoding blocks of SHSnet are inverted residual blocks; we first widen them with a 1 × 1 convolution, then use a 3 × 3 depth-wise convolution to greatly reduce the number of parameters, and then again use another 1 × 1 convolution to reduce the number of channels so that input and output can be added, leading to a residual structure similar to EfficientNet [46]. These blocks are divided into 5 groups (ENC 1-5) that encode the image at 5 different scales, as shown in Figure 1.

DECs: The DECs’ role is thus to up-sample the low-resolution segmentation map to a resolution that is the same as that of the input resolution. This will ensure that each pixel in the input layer will have a corresponding pixel in the output layer. The decoding unit consists of a transposed convolution layer, as shown in Figure 1.

Large RFs: Conventional deep learning models have small RFs, even when they are built deeper [47] (Figure 2A). A conventional solution to circumvent this issue is to use a large input image at the cost of high computational power, which would require expensive hardware. However, when the input image size is lower, this affects the segmentation accuracy (e.g., in

Table 2. Effect of model components on performance.

Components				Performance
Encoder	GCN + BN	Decoder	Loss Function	Accuracy	IOU	BF Score	Parameter × SHSnet
ENC	No	Bilinear Interpolation	NA	0.53	**	**	0.9×
ENC	No	Nearest Neighbor Interpolation	NA	0.52	**	**	0.9×
ResNet-18	Yes	Yes	LF	0.78	0.72	0.75	2.3×
ResNet-50	Yes	Yes	LF	0.84	0.79	0.81	5×
MobileNet	Yes	Yes	LF	0.74	0.71	0.73	0.7×
VGG-16	Yes	Yes	LF	0.72	0.69	0.70	>10×
ENC	No	Yes	LF	0.81	0.74	0.74	~0.95×
ENC	Yes	Yes	Cross-Entropy	0.66	0.51	0.54	1×
ENC	Yes	Yes	LF	0.88	0.81	0.84	1×

NA: Don’t need a loss function for segmentation and IOU and BF score. **: Can’t be calculated.

), especially for complex cracks. Our method solves these problems with a large kernel that will increase the effective receptive field (RF) (Figure 2B), thus facilitating segmentation [47]. To do so, a global convolution network (GCN) module is employed. In the GCN, CNNs are stacked in configurations 1 × K + K × 1 and K × 1 + 1 × K, and finally, the output of these parallel units is added, as shown in Figure 1. This enables dense connections within a large K × K region in the input map, i.e., ‘large kernels’. Compared to the conventional ‘k × k’ convolution, the GCN module requires significantly fewer parameters and thus is more practical in our case.

BNs: The BN module is a residual structure. More specifically, the output refined the scoring map S_out as S_out = S_in + R(S_in), where S_in is the coarse input score map and R(·) is the residual branch. It helps with improving the delineation of cracks from the background.

WMA: Residual-like connections for ‘attention’ were used as shown in Figure 1 to capture long-range features across multiple scales and boost feature representation capabilities; the connections are designed uniquely (as shown in Figure 1) so that attention is weighted across scales, giving more importance to deeper features.

To summarize, SHSnet performs the parameter-efficient deep encoding of images to generate complex features to distinguish complex cracks from the background. The large kernel size allows SHSnet to have a large receptive field. This enables dense connections between various spatial features, leading to the superior per-pixel classification of these encodings. These features are refined to create better delineation between the crack and the background. This process is repeated at different scales and added in residual-like connections to ensure that feature spaces during encoding and upscaling are not lost. These modifications lead to superior performance with a significantly lightweight design, as will be discussed later.

2.1.2. Loss Function (LF)

The use of appropriate loss functions that can guide in ‘learning’ the features correctly is important. An LF is a type of region-based loss function—the focal Tversky loss function.

In our case, due to the thinness of microcracks, the average area of each crack is less than 0.01% of the total area of the specimen. Thus, a region-based loss function (LF) based on the Dice Coefficient (DC) [48], Equation (1), is employed to maximize the overlap between the segmentation map of these cracks and the ground truth. For this problem, the impact of the incorrect positive assignment of a pixel (${\rho }_i^{\widehat{c}}$ $G_i^{\widehat{c}}$) and the negative assignment of a pixel (${\rho }_i^c$ $G_i^c$) is not equal. With this consideration, this LF is used with β as the weighing factor for the relative importance of ${\rho }_i^{\widehat{c}}$ $G_i^c$ and ${\rho }_i^c$ $G_i^{\widehat{c}}$ and probabilistic network output (with probability (${\rho }_i$) of the i^th pixel). In the PLF, irrespective of the number of pixels, each class is treated equally.

$$LF={\left(1-{\displaystyle \frac{\sum _{i=1}^N{\rho }_i^c{G}_i^c}{\sum _{i=1}^N{\rho }_i^c{G}_i^c+\left(1-\beta \right)\sum _{i=1}^N{\rho }_i^c{G}_i^{\widehat{c}}+\beta \sum _{i=1}^N{\rho }_i^{\widehat{c}}{G}_i^c}}\right)}^{\gamma }$$

(1)

Here, N is the total number of pixels in the image, G is the ground truth of the image, c corresponds to class c, and $\widehat{\mathrm{c}}$ corresponds to not being in a class.

With focal parameter γ [49], the change in the LF for the unit improvement of accuracy for pixels at different accuracies was explored, as shown in Figure 3; γ was selected such that the change in the PLF has higher values when accuracy is lower to facilitate the detection of more difficult regions in the HRCF network.

2.2. Post-Processing Unit (PPU) for Crack Morphological Parameters

In the literature, estimating crack parameters from segmentation maps relies on mathematical models based on pixel connectivity. To avoid such errors in the literature, crack reconstruction methods based on image processing models are utilized, such as brightness [9], region connectivity [19], and the TuFF algorithm [50]. They are applied for separated large cracks; thus, they may not always work, especially when cracks are complex. To solve this problem, we present a novel approach that relies on the physics of crack formation and scanning lines. The implementation details are discussed below.

For each input image ($I_{rgb}$) with SHSnet, a binary segmentation map ($I_{bin}$) of the labels for each pixel was generated. This is described in Equation (2).

$$I_{bin}=argma{x}_i\left(SHCnet\left(I_{rgb}\right)\right)$$

(2)

where i is the i^th label.

Each independent crack segment was fitted with an ellipse as in Figure 4a. Then, the equation of the axes of this ellipse was computed. Finally, the orientation of the major axis of this ellipse was computed; then, the direction of the crack propagation as shown in Figure 4a is this orientation.

The fracture model for concrete suggests that the crack propagates in the direction perpendicular to the effective tensile load (ETL). Therefore, the direction that is perpendicular to the average of the orientation of the major axes of all the cracks is taken as the tensile loading direction. With this, a very efficient technique to compute crack parameters such as crack number, crack width, maximum crack width, and crack length from the segmentation map is proposed as follows.

Scanning lines parallel to the direction of the ETL (shown in Figure 4b) were generated digitally. The number of these scanning lines will determine the density of the measurements of the crack parameters. In the binary image, ‘0’ refers to the background and ‘1’ refers to the crack. The cracks are identified by finding the number of step-up ($N_{step-up}$) locations along the scanning line. The crack number (n) is the average of the $N_{step-up}$ of all scanning lines (Equation (3)).

$$n={\displaystyle \frac{1}{m}}{\sum }_{j=1}^m{N}_{step-up}^j$$

(3)

where j is the j^th scanning line.

The average crack width (ACW) (Equation (6)) is computed by averaging the widths ($W_k^j$ = crack width at the kth intersection of a crack segment by j^th scanning line).

$$ACW={\displaystyle \frac{1}{m\mathrm{*}o}}\mathrm{*}{\sum }_{j=1}^m{\sum }_{k=1}^o{W}_k^j$$

(4)

The maximum crack width (MCW) is the maximum of all the Ws (Equation (5)):

$$CW=\underset{j}{\max }\left(\underset{k}{\max }(W_k^j\right))$$

(5)

Since each crack is a thin element for such an element, crack length (CL) is estimated as half of the perimeter of the crack (Equation (6)). I_bin was shifted one pixel along the loading direction and subtracted from the original binary image. Morphological operation erosion was employed to convert the boundaries of the subtracted image to a single-pixel width. Then, the perimeter of the object (i.e., crack) was calculated based on region connectivity, which is a well-known technique utilized in computer graphics [51]. It should be noted that when two adjoining pixels are connected in the vertical, diagonal, or horizontal direction, they are considered part of the same object.

$$CL={\displaystyle \frac{perimeter}{2}}$$

(6)

3. Data Collection and Training Parameters

As explained before, collecting a large enough dataset of complex cracks with diverse material preparation techniques takes years. The experiments here aim to demonstrate the applicability of the proposed methodology on complex (micro)cracks. Moreover, the demonstration of the performance of the proposed method on a more complex scenario will entail that the capability of this technique in other less complex cases is not adversely affected. With these considerations, a dataset is developed through a combination of laboratory tests and field datasets depicting damage to buildings and pavement. The details of the field dataset are discussed further. More specifically, the laboratory dataset consists of images that depict surface cracks of damaged strain-hardening cementitious composite (SHCC) samples through various accelerated damage tests performed in laboratory conditions using our prior experience. What sets SHCCs apart are high-density (up to ~300/m), tortuous (micro)cracks with widths of ~60 microns and the high range of these crack features (HRCF) observed in tandem [9,52]. These values are several orders more complex. For example, cracks in typical concrete are in orders of a few millimeters (100 times higher); similarly, the crack density in the field dataset is ~10× lower.

These datasets were collected sporadically over a long period of time. As a result, the imaging devices, lighting conditions, surface characteristics, and artifacts are different. The dataset consists of 1078 images taken of ~70 different samples. The original images were then processed to compute the expected output (also known as ground truth) for training. This is described in the following section.

Training details should also be mentioned. The total dataset was divided into training, validation, and test sets with the following ratio: 0.7, 0.1, and 0.2, respectively. During training, the images were cropped randomly into the resolution of 224 × 224 × 3 and trained in a batch size of 32. The Adam optimizer with a learning rate of 1 × 10⁻³ was employed in this study to update the weights of the network after each iteration. Training was deemed complete when for next the 10 epochs no improvement in network performance was observed. NVIDIA GPU-1080 was employed to train the neural network.

The network was trained for 120 epochs. The following performance metrics were quantified: accuracy, IOU, and boundary F1 (BF) score. The accuracy measures how well each pixel is classified, IOU measures the similarity between the predicted cracked area and the original area, and the BF score is a measure of how well the predicted boundary matches the object (crack) with the known boundary. Equations (7) and (8) are general equations describing accuracy and BF score. The average values of these metrics are 0.88, 0.81, and 0.84, respectively, on the test set.

$$accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(7)

$$BF score={\displaystyle \frac{2\times precision\times recall}{precision+recall}}$$

(8)

Ground Truth Development

The training set consists of original images (such as Figure 5a) and expected output, i.e., ground truth (Figure 5e). As SHCC cracks are complex and numerous, labeling each pixel manually would require an overwhelming effort. A novel stage-wise labeling technique was introduced to reduce human effort. The details are as follows. At first, the image in which cracks were present was roughly brushed out manually (Figure 5b); then, the area within the brushed region was binarized based on the optimal color filter selected manually (Figure 5c). However, such a binary image is often populated with dot noise, which was removed through region-based filters (Figure 5d). Such an output looks reasonably good except for very thin sections of the crack. These sections were again manually improved to create the ground truth (Figure 5e).

4. Results

4.1. Ablation Study

SHSnet is composed of four modules, encoder, GCN and BN, decoder, and PLF, as described in Section 2. In this section, the effect of various modules such as encoder, GCN and BN, decoder, and loss function on the overall performance is investigated. To study the effect of the encoder’s backbone on various state-of-the-art (SOTA) CNN models, VGG [53], ResNet [54], and MobileNet [30] are used as encoder-trained models. After training was complete, their performance was measured on the test set. This result is reported in Table 2.

To study the effect of the decoder on the overall performance of the network and GCN and BN, the decoder unit was replaced by the conventional image up-sampling technique. In this case, two conventional methods, Interpolation and Nearest Neighbor Interpolation, were utilized. In both these cases, EfficientNet was employed. The results of this study are also reported in Table 2. It shows that the conventional method for up-sampling the encoded results does not create accurate segmentation maps because these sections of the model do not learn to produce a sharp segmentation mask.

Furthermore, to study the impact of the GCN and BN together, two cases are studied: (a) SHSnet and (b) SHSnet without GCN and BN. The result of this study is also reported in Table 2. The result confirms a large increase in IOU and BF scores, which confirms that the addition of the GCN and BN results in an improvement in identifying and creating sharp boundaries.

To measure the effect of the loss function on the overall performance of SHSnet, two cases were studied: (a) SHSnet and (b) the same network architecture of SHSnet with the loss function replaced with a conventional loss function. The conventional loss function, cross-entropy (CE) loss, has been widely used in classification tasks (such as ours, in which each pixel is classified as a crack or background) in AI applications. The proposed loss function gives equal weight to each class, whereas the CE loss function gives the result of each pixel equal weight. These values are significantly lower than the optimal values (0.88, 0.81, and 0.84, respectively). This is due to CE loss giving equal importance to each pixel; thus, in a dataset that mostly consists of pixels that are not cracks, even if the cracks are misclassified, CE loss remains low and cannot guide the network to appropriately learn the features representing cracks in SHCCs. This behavior is evident in a typical qualitative result reported in Figure 6, which shows that in both cases it is possible to learn the pixels corresponding to NC equally well; however, with CE loss, even when all the parameters are kept constantly trained, the network fails to correctly identify the cracks.

4.2. Comparative Analysis of SHSnet

In this section, the performances of various deep learning models are compared. For this comparison, the network was selected based on different architecture or encoder characteristics to cover a wide range of cases. For the VGG-based model, Typical-RC-AI was selected, as this has been shown to work for a wide range of cases. For the Unet-based model, CrackUnet (7…15), which was specifically designed for crack segmentation, was selected. From DeepLabV3+ architecture, Sarmiento’s Model and Song’s Model were selected. From the SegNet model, CrackSegnet was selected. The standard models FCN and SHSnet were compared on a crack dataset collected from laboratory tests. Note that the number with CrackUnet denotes the depth of the encoder. Their performances are reported in

Table 3. Comparative performance on SHCC microcracks.

Deep Learning Model			Performance
	Accuracy	Precision	Recall	IOU	BF Score	Param.
Typical-RC-AI	0.66	0.74	0.46	0.56	0.56	34M
CrackUnet7	0.71	0.51	0.64	0.59	0.63
CrackUnet11	0.78	0.58	0.71	0.71	0.72
CrackUnet15	0.82	0.70	0.84	0.77	0.78	30M
DeepLabv3+	0.78	0.79	0.72	0.74	0.76	16M
Song’s Model	0.74	0.75	0.67	0.71	0.73	9M
FCN	0.62	0.53	0.56	0.53	0.51	132M
CrackSegnet	0.76	0.75	0.76	0.64	0.67	15M
SHSnet	0.88	0.85	0.83	0.81	0.84	4M

.

It was observed that fully convolutional neural networks such as FCN produce one of the lowest scores. Models based on DeeplabV3+ use ASPP units to improve segmentation, leading to higher performance. As compared to them, CrackSegnet and CrackUnet have better performance; this can be attributed to attention modules and other specific changes in the network design. In comparison to these models, SHSnet has superior performance. This could be due to the large RF + BN, which allows a superior understanding of the context and leads to higher accuracy, especially when the input size is small.

It was observed among these deep learning models that the proposed network has at least 10x fewer parameters than the second-best-performing models; thus, it is more efficient. The results also highlight that, probably due to the appropriate design of SHSnet to deal with the additional complexity of the cracks along with the high imbalance in the dataset, SHSnet has shown to perform superior in segmenting SHCC cracks. A comparative example is also shown in Figure 7. This result also reinforces this behavior.

4.3. Performance of PPU

Accuracy: These binary images were further analyzed to estimate crack width, crack number, and crack length using the PPU. To evaluate the performance of these computed characteristics, the original images were also analyzed manually to compute these crack characteristics. An example of the results of crack features measured during testing is shown in Figure 8, the PPU shows that the relative error is 0.4%, 1.2%, and 5.6% for crack number, length, and width, respectively when compared to those computed manually.

Efficiency: To understand the impact of the proposed technique, the total time it takes to estimate various crack parameters by (a) analyzing the image through our technology and (b) analyzing images with the current state of the art (i.e., with crack scope) was monitored.

Table 4. Comparative analysis of computational efficiency.

Computing With	Crack Number	Crack Width	Crack Length
Proposed Model	0.3–5 min/image *
Manual	~6 min/image	~200 min/image *	~100 min/image

* Computation on 1060 GPU 7th Gen CPU.

reports our findings for a typical 20MP image. It is observed that the proposed method significantly (~90%) reduces the time required for computing various crack parameters.

5. Applications

The transfer of deleterious materials within concrete is associated with the durability of the concrete in buildings (ACI 224R) as well as in bridges (AASHTO). The transfer of deleterious materials could take place by diffusion. The diffusion coefficient ($d$) of a cracked specimen is a function of the product of crack width ($cw$), crack length ($cl$), and crack number (n), as shown in Equation (9).

$${\mathrm{d}}_{\mathrm{c}}={\mathrm{K}}_{\mathrm{a}\mathrm{v}\mathrm{g}}\mathrm{c}{\mathrm{w}}_{\mathrm{a}\mathrm{v}\mathrm{g}}\mathrm{n}\mathrm{c}{\mathrm{l}}_{\mathrm{p}\mathrm{e}\mathrm{r} \mathrm{c}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{k}}$$

(9)

Here, n is the number of cracks.

It is expected, according to the theory, that (A) crack morphological parameters increase with strain [55] and that (B) the diffusion coefficient ($d_c$) increases in polynomial form [56,57,58]. To test this, the PPU was utilized to monitor crack parameters and d in an accelerated test. Figure 8 presents the result of the evolution of these cracks’ morphological parameters and d computed from Equation (9) with the proposed technique at different strain values, which validates the theoretical results. This behavior is also consistent with the behavior of metals under fatigue [59,60,61].

The physical test results match each other well, which validates both the theoretical and practical applications of the proposed methodology in predicting the transport properties of ECCs.

The physical test results match each other well, which validates both the theoretical and practical applications of the proposed methodology in predicting the transport properties of ECCs.

5.1. Generality across Multiple Materials

The existence of a crack indicates that a (construction) material is stressed beyond the allowable limit. In almost every situation, cracks are observed to form a pattern. Scientists and researchers analyze these patterns to understand the cause and design processes to avoid such damage. Visualizing these crack patterns allows researchers to decipher the nature of underlying stresses, i.e., forensics of failure. Therefore, understanding crack patterns is of utmost importance. However, in construction, a lot of different constituent materials are introduced to replace concrete, and they have different colors. Color and texture are also expected to change based on construction practices and the age of the structure. A technique that can incorporate more wider and complex cases is expected to perform well by incorporating these cases.

To test generality, additional test conditions were designed to replicate the above cases. These samples were selected to encompass possible variations in surface preparation, constituent materials, and crack properties (different densities and widths). This spectrum was divided into multiple classes, as reported in

Table 5. Performance of SHSnet on different types of concrete.

Experimental Condition			SHSnet	Conven.
Surface Preparation?	Surface Color	Crack Density	Accuracy	Accuracy
Yes	NA	NA	0.924	0.609
No	Light	Medium	0.889	0.544
No	Light	High	0.891	0.521
No	Dark	High	0.874	0.504
No	Dark	Medium	0.869	0.513
Images From Internet/Literature			0.853	0.439

NA: not accounted.

. Some details should be discussed: the samples were divided into prepared, i.e., painted or unprepared cases; ‘high’ and ‘medium’ in crack density with the threshold of ~150 cracks/m; and ‘light’ and ‘dark’ based on surface color. Finally, another class was created: it consists of a group that shows complex cracks taken from the literature. These images are from the experimental results of various labs where the pre-processing of the images and data collection are unknown. Some representative photographs of SHCC microcracks with various types of surface preparations, surface colors, and crack densities and the segmentation results are shown in Figure 9. The performance metrics of SHSnet in these experimental settings are reported in Table 5. The results in Table 5 show that, unlike the conventional technique, the proposed method is not sensitive to a single surface color. The proposed method performs equally well in cases with ‘medium’ crack density (thus fewer intersections) and cases in the sample that have a ‘high’ density (with a large number of intersections). Images from the internet are not collected for further processing of cracks and thus have low-resolution large noise; markings on top of the cracks especially after submission to a journal are compressed, which enhances the adverse effect and leads to substantially poor performance in a conventional method. Thus, Otsu’s method performs poorly. In such a case, the SHSnet accurately segments the cracks.

To summarize, in all such cases, even when the surface is unprepared and dark with high density (which even includes thin cracks propagating through surface defects, as in Figure 6), the segmented cracks, i.e., the crack pattern, match well with manual observations.

5.2. Inspection of Roads and Buildings

To demonstrate the application of this method, two scenarios are presented: (1) inspection of buildings and (2) inspection of roads. To achieve this, the proposed method is applied. The following two datasets were selected:

(1)

DeepCrack: This is an online dataset showcasing the cracks on asphalt and concrete road surfaces with many different surface textures. The dataset was collected by Liu et al. [22].

(2)

Concrete Crack Dataset (CCD): This dataset was collected in China, showcasing cracks in different sections of buildings. The data were collected by Yang et al. [19].

For these datasets, the performance indicators from the segmentation results are reported in

Table 6. Comparative performance of SHSnet on real-world dataset.

Method	Parameters	CCD			DeepCrack
		Pre	Rec	F1	Pre	Rec	F1
Typical-RC-AI	34M	0.81	0.78	0.79	0.73	0.76	0.75
DeeplabV3+	16M	0.79	0.75	0.77	0.74	0.68	0.72
FCN	132M	0.71	0.71	0.71	0.67	0.67	0.67
CrackUnet	32M	0.91	0.86	0.88	0.87	0.83	0.86
CrackSegnet	15M	0.85	0.87	0.86	0.82	0.79	0.82
Ours (SHSnet)	4M	0.95	0.96	0.95	0.96	0.93	0.94

Apart from our method, this was compared with other methods from the literature. A deep convolutional network based on convolutional layers of VGG16 (similar to Yang et al. [19]) was used for comparison. This network is referred to as Typical-RC-AI. CrackUnet, which was developed from Unet, has shown superior performance over other deep learning models, such as the fully convolutional network (FCN) [62] and Sarmiento’s Model (Sar’s Model). The results consistently show that our proposed method performs with high accuracy for datasets showcasing real engineering infrastructure. Not only this, but when comparing the results with other models in the literature, our model requires a significantly smaller number of parameters to achieve this. These results validate the potential of the proposed technique in field applications.

6. Conclusions

The characterization of cracks becomes convoluted due to inherent complexities such as (interactions from) high-density cracking, thin crack widths, and the tortuosity of cracks. In SHCCs, these complexities are inherent and occur together.

In this work, a novel deep learning model for the end-to-end segmentation of thin, tortuous, and dense microcracks in diverse conditions is developed. SHSnet is a weighted, ‘attention’-based, highly efficient, large receptive field, boundary-refining deep learning model that is strategically designed for the end-to-end segmentation of complex cracks. SHSnet achieves accuracy, IOU, and BF scores of 0.88, 0.81, and 0.84, respectively.

A novel region-based loss function is adapted from the focal Tversky function, which gives equal class weightage and unequal weightage to easy and complex regions embedded in the network. This ensures that the network consistently learns the crack features from a highly unbalanced dataset and that the performance is not adversely affected by a high range of crack feature (HRCF) complexities.

A PPU was developed for crack morphological parameter calculation from incomplete/inaccurate segmentation maps using the physics of concrete fracture and crack geometric properties and the scanning lines derived from it. The relative error observed for crack number, length, and width is 0.4%, 1.2%, and 5.6%, respectively.

A comparative analysis shows that, in general, using attention and proper design improves crack segmentation as compared to the vanilla model. SHSnet, which adds an LF and a large receptive field apart from attention, performs superior to models in the literature while requiring significantly lower parameters with similar capability.

By applying the SHSnet on samples with different textures, color (dark to light) crack density (up to ~300/m) widths (30–300 μm), and image sources, it was possible to show that there is no bias in the proposed network for certain types of systems. In all cases, segmented cracks match well with the original picture. Beyond the result, a novel way to test generality by using laboratory samples in accelerated testing was explored. This could be an alternative method for data generation, but it needs to be explored further.

The proposed methodology can also be applied to test the evolution of crack characteristics relevant to durability, as it matches well with those computed manually during the inspection of roads and buildings.

In the near future, this method will be utilized on cracks generated from different tests, like fatigue and bending tests, and in different materials, such as metals.