CNN-Based Automatic Mobile Reporting System and Quantification for the Concrete Crack Size of the Precast Members of OSC Construction ^†

Abstract

Civil infrastructure over the years has experienced a dominant reliance on concrete material compared to other construction materials. Human inspection is the main mode of inspection for such structures, including concrete columns, which has been proven to be inaccurate and time-consuming. Convolutional neural networks (CNNs) are a substitute for such problems for both detection and quantification. However, storing the results and visualizing them at a later stage has always been a challenge. Additionally, integration of the concrete crack deep learning model to a mobile platform is an area that has received less attention. This study focuses on segmenting the concrete crack sections using the latest state-of-the-art (YOLOv7) neural network, which is then used to obtain the quantification data about the length and width of the detected crack using image binarization, and finally the results are published using a reporting system integrated to a mobile platform using a web and IoT system. The published report uses a checklist from the quantification results to grade the crack as well as its structure. The results show a mAP of 0.85, while the quantification results show a 10.82% absolute error, respectively. The reporting system takes a combined average of 5940 ms to store the data inside a database, which is then published through a mobile device. It has been demonstrated through this study that an automatic mobile reporting system is feasible to be used on buildings for maintenance, which can be further applied across other sectors of construction for monitoring and repair purposes.

1. Introduction

Concrete as a construction material is heavily relied upon in the construction industry. In building construction projects there is a sixty percent usage of concrete as a material on average compared to other construction materials [1]. In civil engineering construction, concrete plays a critical role in the structural integrity of the structure. Hence, the safety of the majority of these civil engineering structures is determined by the quality of the concrete [2]. However, concrete structures are subject to defects over time, with one major defect being concrete cracks. The structural health of a concrete structure is mostly determined by the size of the concrete cracks. Large concrete cracks tend to expose the concealed structural steel members undermining the structure’s overall integrity. The most dominant mode of monitoring concrete cracks is manual inspection, which on the contrary has been cited to be time-consuming and at the same time prone to errors [3].

The proposed system in this study uses a convolutional neural network, which is a computer vision algorithm to detect the location of these cracks and segment the crack regions. The segmented crack regions are then processed to a binarized format to obtain the crack length and width in a pixel format. Sinha (2006) briefed about the development of a statistical filter for the detection and segmentation of cracks in concrete pipes [4]. Yang (2018) used a system to identify cracks using the pixel segmentation technique [5]. Lee (2010) proposed a crack measurement technique on the length, width, and crack orientation based on the artificial neural network [6]. Kim’s (2019) system was based on a mask RCNN model to measure the concrete crack width but not the concrete crack length [7]. Yuan (2021) used a convolutional neural network to measure the crack length and monitor the crack propagation, however the system did not measure the crack width [3]. Licun Yu (2022) detected concrete cracks using bounding boxes and measured the width of the cracks leaving crack length measurements [8]. Detection and segmentation have been discussed in the past, but presenting these results and using it for automatic reporting has rarely been discussed for the structures in these previous studies. Sohn (2004) put forward a new crack monitoring tool, CCMS, with a graphical user interface for crack analysis and detecting the arising changes in these cracks using images [9]. Shuang-rui (2015) detected and segmented the crack width using an android device application that had a user interface for saving the results [10]. Lee (2022) gave a deep learning-based system for the quality inspection of cracks on the concrete members [11]. Zhang (2022) proposed a bridge surface crack detection method and displayed it using a mobile device and a server without measurement parameters [12], while Li (2023) devised a system for the identification of the damage in bridge structures and displayed the evaluation results using commercial software [13]. However, all the reviewed studies were not based on a mobile system or reporting system of the quantified concrete cracks.

2. Methodology

In the fields of computer vision and construction, the instance segmentation technique has recently been employed. The capability of the model to perform real-time segmentation using neural networks with high accuracy makes it efficient for detecting the required content.

The model development began with the collection of data images containing concrete cracks. In this process, a total of 200 images were collected. Then, to make the model more robust in differentiating between the concrete cracks and concrete crack noise the images were cropped to only those sections that contained concrete cracks. The data was increased from 200 images to 1000 images through five image augmentation techniques: 50% horizontal flip, gaussian blurring, image recoloring, image translation, and image rotation. Data obtained through image augmentation and the original data were resized to a uniform size of 224 × 224 × 3, where these dimensions indicate the length, height, and channels of the image data, respectively. The image data with their corresponding annotated files were passed through the instance segmentation, and a model was obtained which could make inferences to the new concrete crack data. The hyperparameters of the trained model involved a batch size of 16,400 epochs, an input image size of 224 × 224 corresponding to the length and width, an initial learning rate of 0.01, and one class (Crack). The internal structure of the model is composed of 104 convolutional neural network layers with 5 max-pooling layers using a stride of 1 and the concatenation layers which were used for accurate feature extraction. The overall model contained the input, the backbone for feature extraction, the head for refining the feature maps, and the final segmentation, as shown in Figure 1. The segmentation was performed in the final convolutional layers 102–104.

The system’s architecture, as shown in Figure 2, includes a Flask API platform, which is connected to a mobile device and a computer server. Upon an HTTP request, the captured image from the mobile device is segmented onto the server, which stores it and then masks it using the image binarization technique, after which the nonzero mask pixels are counted along with the image size parameters to calculate the length, which is then divided by the total pixel values and halved to obtain the value of the width and classify it according to the criteria proposed [12].

3. Conclusions

From the results shown in

Table 1. Relative error percentages for the length and width of the proposed reporting system.

Crack	Proposed System Result	Hand Measurement	Error (%)
Length (mm)	49.66	52	4.5
	51.07	58	11.95
	52.60	60	12.33
Width (mm)	0.48	0.39	23.08
	0.73	0.67	8.95
	1.20	1.24	3.23

, as the width of the crack decreases the absolute error of the proposed system also increases. However, for the crack length, the measured length and the actual length of the crack show a minimal error, with the highest absolute error obtained from the test being twelve percent. While maintaining a fixed distance of 10 cm from the concrete crack object to the camera device, the system was able to quantify the length of the crack with an average of 90% percent accuracy and the width with 88% accuracy, respectively, as stated in Table 1. The hand measurement of the crack length was performed with a ruler, while for the width, a vernier caliper was used to accurately measure the width of up to 0.1 mm.

The inference time for the concrete crack quantification was 5940 milliseconds from the image uploading to the final reporting of the quantified results. The quantified width has been used in the classification of the crack in the checklist, which ultimately serves as the basis for damage in the member whether or not it should be discarded on the site. The system was assessed on static images, but the limitation of the system was that the distance between the camera and the concrete crack object had to be made constant by 12 inches from the crack for testing, and the training images and scale had to be set accordingly by equating the pixels to mm. Hence, a modification to the system to handle variable distances is considered as an area of further research. Moreover, the dataset should be updated to further enhance the accuracy of the system.