**1. Introduction**

Sensing and quantifying damage plays a critical role in the process of structural health monitoring, which aims to detect structural damage and provide early warnings when a possible risk of failure is detected. Many structural health monitoring systems employ accelerometers, displacement sensors, or piezoelectric sensors located at selected locations to monitor changes in the structure's deformation, natural frequencies, and modal shapes [1,2]. These systems then evaluate possible failure modes, damage levels, and locations. While accelerometers are typically employed for beam-column-based structures such as buildings, these are not the optimal sensors for structures whose failure modes are insensitive to the structure's vibration characteristics. For some types of structures such as dams, tunnels, and reinforced concrete vessels, or shear-critical components such as reinforced concrete (RC) walls, the detection and evaluation of cracks is a relatively practical approach for safety assessment and monitoring.

Several structural damage indices have been proposed. Park et al. [3] proposed a damage index for a structural system according to its largest system displacement, ultimate displacement, accumulated strain energy, cyclic loading effect, and system yield force and displacement. Based on the calculated damage index, the structural system can be classified into one of the following damage levels: slight, minor, moderate, severely damaged, and collapsed. Roufaiel and Meyer [4] proposed a damage index that uses the initial stiffness, current stiffness, and failure stiffness. Powell and Allahabadi [5] proposed an index based on the current displacement, yield displacement, and ultimate displacement. These damage indices consider a structure as a single-degree-of-freedom system to simplify damage level estimations. However, in practical applications, these damage indices are difficult to use, as the stiffness and the displacement of a structure is sometimes difficult to measure for real, multiple degrees of freedom, and partially damaged structures. Detailed structural performance and safety may require advanced structural analyses based on finite element analysis tools [6,7] or structural experiments [8,9] which are specific to a certain type of structure. For the purpose of structural health monitoring, the displacements of certain locations can be monitored by pre-installed displacement devices; however, current stiffness and other structural properties are difficult to accurately measure or estimate.

Alternatively, for easy to implement and quick structural safety assessments of reinforced concrete (RC) structures, several evaluation methods have been proposed that instead consider the surface cracks of concrete structures. The Japan Building Disaster Prevention Association (JBDPA) provides a guide based on the visible cracks in the concrete surface of beams, columns, or walls, and categorizes damage into five classes according to the maximum opening width of the cracks [10]. According to the JBDPA criterion, structures with a maximum crack width larger than 0.2 mm, 1 mm, and 2 mm are categorized as showing light damage, moderate damage, and heavy damage classes, respectively. The International Atomic Energy Agency (IAEA) uses a more conservative standard that categorizes cracks with an opening width larger than 0.2 mm and 1 mm as moderate and severe damage, respectively [11]. The bridge inspector's reference manual, published by US Department of Transportation [12], categorizes cracks into structural cracks, flexural cracks on a tee beam, shear cracks on a slab, temperature cracks, shrinkage cracks, longitudinal cracks, etc.

For surface damage detection and evaluation, image-based measurement is an automatic and cost-efficient method in terms of hardware cost. As the aforementioned structural health monitoring or damage detection methods have different features, advantages, and limits, no single method can be used to replace another, nor can it be used as the sole means of structural health monitoring or damage detection. Image-based measurements, and their potential for damage detection, are not intended to replace any of the aforementioned methods. Instead, the image-based method aims to provide an area-based measurement method to measure or monitor cracks [13], strain fields [14,15], multi-axial displacement [16], or structural vibrations [17], where technology for conventional displacement measurements is inadequate [18]. The hardware cost may be relatively low [19], and may even employ existing surveillance cameras in the structure, thus eliminating the need to install additional cameras [20]. With recent dramatic improvements in digital image processing techniques, image analysis algorithms, accuracies, reliability, and computing speed have improved as well; thus, image measurement has a strong potential for practical structural health monitoring applications [21].

This work develops an image analysis-based damage indexing method following a previously developed image-based crack measurement method. This method is tested using two cyclic tests of RC containment vessels [22]. The vessels are shear critical with a large number of shear cracks induced by only a small displacement. A fractal dimension method [23] is modified and employed in this work to quantify the number of cracks. Based on the number of cracks, as well as their opening widths and orientations, a method for calculating damage indices is proposed. This method modifies the previous image analysis method [24], such that concrete surface crack orientations can be determined automatically. In addition, the fractal dimension crack analysis method [23] is modified so that the damage index can be separated into a shear damage index and a flexural damage index to distinguish between the different types of failure. The combination of these methods will make it possible to carry out structural health monitoring in an automatic manner in practical applications in the future. This paper further demonstrates the image measurement and damage indices calculation procedure based on the aforementioned RC containment vessel experiments.

#### **2. Image Measurement of Cracks on Concrete Surfaces**

Image-based monitoring and damage identification consists of two major procedures: image measurements and damage quantification. Image measurements analyze the image(s) of the measurement regions of interest and provide details, such as locations, lengths, opening widths, sliding displacements, and the orientation of the cracks. The damage evaluation procedure estimates the damage level or index of the measurement region according to the analyzed results from the image measurement.

Many image measurement algorithms and methods have been proposed to detect cracks on measurement regions, such as on concrete surfaces or pavements. These methods can be classified into two groups: (1) edge detection-based methods, and (2) displacement field-based methods. Edge detection-based methods are capable of finding cracks that appear as dark lines in an image. The cracks need to be of sufficient width to appear as dark lines, which is theoretically the width of a pixel. Edge detection methods [25–28] or machine learning methods [29–32] are typically employed to identify the locations or widths of cracks. A review of crack detection methods can be found in [33].

Alternatively, the displacement field-based method identifies cracks according to the displacement field of the measurement region, where the displacement field is analyzed by image analysis techniques [24,34]. Due to the high precision of image-based displacement field measurements, displacement field-based methods are capable of detecting thin cracks with widths of much less than one pixel. Yang et al. [34] detected cracks as thin as 0.2 pixels in photos in an outdoor experiment where images contained environmental light noise. The same image analysis technique detected thin cracks whose width was equivalent to 0.03 pixels in photos in a structural laboratory [24]. This type of method estimates the cracks' opening widths, sliding displacements, and orientations, according to the change in the displacement field between each set of photos taken before and after cracks occurred, respectively. Thus, the first set of photos is used as a reference for the displacement field. Compared with edge detection-based methods, displacement field-based crack detection methods are suitable for thin crack detection, monitoring the early stages of crack development, or monitoring large regions where pixels are relatively coarsened. However, it should be mentioned that most edge detection-based methods used are tailored for inspection, rather than health monitoring. They are more suitable for that purpose than displacement field methods. In addition, displacement field methods tend to be more computationally expensive.

This work employs a displacement field-based method for crack measurement. However, this does not mean that edge detection-based methods cannot be applied to the damage evaluation method proposed in this work. The displacement field-based method is employed here because it is capable of detecting thin cracks that occur in the early stages of structural damage. In addition, the image measurement software, ImPro Stereo, is publicly available on the internet [35], and is further integrated with the damage evaluation computer codes developed in this work.

The displacement field-based method for crack measurement includes five main steps: camera calibration, measurement region positioning, metric rectification, displacement field analysis, and crack analysis. A detailed procedure can be found in [34,36]. This work only focuses on the analysis results as related to the follow-up damage evaluation procedure which is proposed herein.

Camera calibration is the process of finding the intrinsic and extrinsic parameters of the camera. The intrinsic parameters are essentially its optical properties, such as the fields of view and optical distortion coefficients. The extrinsic parameters describe the camera position and its orientation. Typically, the camera calibration process is only carried out once, by taking more than 10 pairs of photos of calibration objects (such as a chessboard of known size) during camera installation (see Figure 1).

**Figure 1.** Stereo calibration of two cameras.

Measurement region positioning tracks the updated position of the measurement by precisely tracking the 3D positions of the control points that are used to define the measurement surface. Defining an ideal planer rectangle measurement requires at least three control points, while a cylindrical measurement region requires at least four, as shown in Figure 2. The positions of control points P1 to P4 describe the movement and deformation of the overall measurement region. Details of the process can be found in [24].

**Figure 2.** Measurement region positioning by tracking control points.

The image rectification process generates a rectangular image that represents the image pattern on the measurement region. The perspective and lens distortion effects are removed during this process. The metric rectified image can be seen as an expanded planer surface of the measurement region so that the ratio of a pixel to its physical length is constant over the entire measurement region; thus, it is essentially an image that represents the unfolded plane from the measurement region. The constant pixel-to-physical length ratio is an important property for the subsequent displacement field-based crack analysis. The rectified image is generated pixel-by-pixel, while the image intensity of each pixel is estimated by mathematically projecting a 3D point onto the surface to its image position in the photo according to the intrinsic and extrinsic parameters of the camera. Its image intensity is acquired through the numerical interpolation of neighboring pixels, as shown in Figure 3.

**Figure 3.** Metric rectification of the region of interest on a cylindrical structural component.

The displacement fields of the measurement region can be estimated by comparing the initial and current rectified images (see Figure 4a,b) using an object tracking method, such as template matching, digital image correlation, an enhanced correlation coefficient, or the optical flow method. Details of the process can be found in [24]. The example presented in Figure 4 was obtained from an experiment that had a measurement region of approximate dimensions of 1.4 m × 0.9 m. Each rectified image in Figure 4a is approximately 2400 × 1600 pixels. The displacement field in Figure 4b is a vector field with 90 × 60 cells, that is, each cell is represented by a sub-image with a size of 27 × 27 pixels (rounded from 2400 / 90 = 26.67). The crack opening in Figure 4c is a scalar field with the same refinement. The refinement is assigned by users, and should be tuned according to the image quality of photos when this method is being applied in practical applications. The displacement field of the rectified images is obtained by optical flow analysis [37]. The resolution of the rectified images and the refinement of the displacement and crack fields are adjusted by the user, and typically depend on the resolution and quality of the experimental photos.

**Figure 4.** Estimating a displacement field by comparing initial and current rectified images. (**a**) Rectified images; (**b**) Displacement fields *u* (*ux* and *uy*); (**c**) Crack opening (*co*).

Crack analysis converts a displacement field to a crack distribution. Crack analysis is suitable for thin cracks that are too thin to display as a dark line in photos, thus requiring the use of the displacement field to estimate the crack's opening width. Each cell of the crack opening width *co* (see Figure 4c) and crack sliding displacement *cs* of any arbitrary cell in the grid is estimated according to the displacement of its four neighboring cells. Crack sliding is the relative displacement of part A with respect to part B, i.e., parallel to the crack orientation. By using the formulation presented in [24], as shown in Equations (1)–(3), the crack distribution can be estimated by a displacement field. The crack analysis method is only suitable for brittle materials such as concrete, as it assumes that the deformation in the displacement field is mainly caused by cracks, rather than strains [34]. In addition, since the image is the appearance of the material surface, it does not represent the crack opening or

sliding under beneath the surface; these are the limitations of this method. The crack distribution is a field of crack opening widths, sliding displacements, and crack orientations. It is discretized to a grid with the same grid density as the displacement. Each cell of the crack opening width *co* and crack sliding displacement *cs* of any arbitrary cell in the grid can be calculated by Equations (1)–(3).

$$
\begin{pmatrix} c\_o \\ c\_s \end{pmatrix} = \begin{pmatrix} \cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{pmatrix} (\mathfrak{u}\_A - \mathfrak{u}\_B) \tag{1}
$$

where

$$\mu\_{A} = \begin{cases} \frac{\underline{u\_{U}} \cdot |\cos \theta| + \underline{u\_{L}} \cdot |\sin \theta|}{|\cos \theta| + |\sin \theta|}, & \text{if } 0 \le \theta < 0.5\pi \\\\ \frac{\underline{u\_{D}} \cdot |\cos \theta| + \underline{u\_{L}} \cdot |\sin \theta|}{|\cos \theta| + |\sin \theta|}, & \text{if } 0.5\pi \le \theta < \pi \end{cases} \tag{2}$$

$$u\_B = \begin{cases} \frac{u\_D \cdot |\cos \theta| + u\_R \cdot |\sin \theta|}{|\cos \theta| + |\sin \theta|}, & \text{if } 0 \le \theta < 0.5\pi \\\\ \frac{u \underline{u} \cdot |\cos \theta| + u\_R \cdot |\sin \theta|}{|\cos \theta| + |\sin \theta|}, & \text{if } 0.5\pi \le \theta < \pi \end{cases} \tag{3}$$

*uU*, *uD*, *uL*, and *uR* are the displacement vectors of the upper, lower, left, and right neighboring cells of any arbitrary cell in the displacement field, respectively (see Figure 5). The orientation of the crack of the analyzed cell is determined by iteratively testing θ within 0 and 180 degrees with a step of 15 degrees (i.e., 0, 15, 30, 45, ... , 165 degrees). To be conservative, the θ which leads to the largest crack opening is selected in this method. If there is no crack on the cell, *cs* and *co* would be very small compared with those with cracks. Small values of *cs* and *co* are caused by either noise, image analysis errors, or relatively small strains, and are ignored in the crack analysis. Figure 4c demonstrates the discretized grid of a crack pattern estimated from its displacement in Figure 4b. It should be noted that the size scale in Figure 5 is only for demonstration. A crack is typically much thinner than the size of a cell. The cracks shown in Figure 4c are actually as thin as 0.02–0.2 mm, i.e., much thinner than the size of a cell in Figure 4b,c. In Figure 4c, the size of a cell is equivalent to a 27 × 27-pixel sub-image. While a 0.02-mm crack can be recognized by the naked eye at a close distance when inspecting damage in structural experiments, it cannot be recognized by most of the edge detection-based methods, as the crack is typically too thin to appear as a dark line in photos. In addition, human inspection is not practical for automatic structural health monitoring.
