Recognition of Damage Types of Chinese Gray-Brick Ancient Buildings Based on Machine Learning—Taking the Macau World Heritage Buffer Zone as an Example

Abstract

As a result of environmental and human influences, several types of surface deterioration emerge on historic buildings, resulting in a decline in the quality of these structures and even threats to their safety. In the conventional approach, assessing the surface damage on a structure involves the time-consuming and labor-intensive judgment and evaluation of trained professionals. In this study, it is suggested that the YOLOv4 machine learning model be used to automatically find five types of damage to historical gray-brick buildings. This would make the job go more quickly. This study uses the gray-brick wall buildings in the buffer zone of the global cultural heritage in Macau as an example. In total, 1355 photographs were taken on-site of the gray-brick walls, and the five most common types of damage were identified. By slicing and labeling the photos, a training set of 1000 images was created, and through 200-generation model training, the model can accurately identify and effectively identify the damage state of the gray bricks and enhance the quality judgment and evaluation of the exterior walls of historical buildings. Experiments allow us to reach the following conclusions: (1) The damage to the gray-brick ancient buildings in Macau is affected by the subtropical maritime climate. Missing paint, stains, and cracks are the main contributors to gray-brick wall damage. (2) Machine learning can help determine the type of damage to old gray-brick buildings, which is useful for managing and protecting historical buildings. (3) The model in this study can identify five types of damage: missing, cracking, plant or microbial erosion, yellowing, and pollution on the exterior walls of ancient gray-brick buildings. It is helpful to accurately identify and evaluate the damaged condition of the gray-brick wall and formulate corresponding protection schemes.

1. Introduction

1.1. Research Background

Chinese gray brick is an indispensable component of traditional architecture. In traditional Chinese architecture, it is more prevalent: for example, the old courtyard of Beijing, the ancient city of Pingyao in Shanxi, and Hui-style dwellings. Additionally, there are preserved historical sites, the majority of which consist of gray-brick buildings, such as the ancient city wall and the Big Wild Goose Pagoda. Clay is used to make gray brick, and the clay contains iron ions and inorganic salts. The iron ion is unstable and will oxidize when fired at a high temperature in the kiln. When the kiln is fired, a large amount of water is poured from the top of the brick kiln. Under high-temperature conditions, a large amount of water vapor is immediately generated to isolate the air. The lack of oxygen causes incomplete combustion of coal, resulting in lower CO levels. In this way, Fe₂O₃ is reduced to black iron oxide and blue-black Fe₃O₄, which form the color of the gray brick. Therefore, the color of the gray brick produced by each batch of brick kilns is not uniform. Due to water splashing and repeated oxidation and reduction, gray bricks are more pressure- and water-resistant and difficult to peel off than red bricks. Consequently, the ancient Chinese gray-brick buildings have withstood centuries of wind and rain and continue to stand tall [1,2].

However, over time, gray-brick-built walls are often affected by factors such as air pollution, flooding, and erosion, causing them to deteriorate and break. More seriously, this may even lead to the collapse of historical buildings. Similarly, Macau is located in the Pearl River Delta of China, which has a subtropical maritime climate, and the local climate conditions are more complicated (Figure 1). Macau has a maritime subtropical monsoon climate. Every year, the weather is at its best and most stable from mid-October to December. The weather is warm and sunny with low humidity, making it the most comfortable. In spring, around March or April, the weather is humid and foggy. From mid-May to early June, affected by the southwest monsoon, it is hot and humid, and there are occasional long-term heavy rains brought by low-pressure troughs; from July to August, it is mostly affected by the southeast monsoon, the temperature rises and the continuous rainstorms decrease (Figure 1), and from time to time, tropical cyclones hit. On the other hand, the World Heritage buffer zone of Macau is mainly around the old city area. In the past, there were a lot of Chinese people living there, and their buildings were mainly made of gray bricks. The areas where they live, most of which are low-lying areas, have undergone reclamation movement many times in the past 100 years. It is extremely vulnerable to salt water soaking in the building’s foundation in the face of global climate change and summer typhoons. thereby further destroying the structure of the gray brick. At present, there are still a certain number of historical buildings with gray-brick buildings, and they are also facing the same protection problem. In particular, the historical buildings distributed in the buffer zone of the World Heritage site in Macau are facing different degrees of damage caused by climatic factors and human factors.

1.2. Literature Review

In the history of ancient Chinese architecture, the appearance of bricks happens relatively early. In the era of “Qin bricks and Han tiles” (B.C. 221 to A.D. 220), the technology of firing bricks and tiles has been relatively mature. However, due to material and economic constraints, gray bricks were not widely used on building sites until the Ming Dynasty (A.D. 1368 to 1644). Therefore, the restoration of gray bricks also evolved from the craftsmen of the Han Dynasty (B.C. 202 to A.D. 220) to become a professional technology. During the restoration process of historical buildings constructed of gray brick, the traditional and common method is to hire professional technicians to conduct on-site investigation and analysis. At the same time, professional equipment is used to identify the surface damage on gray bricks [3,4]. Although this method has significant professional advantages in its operation, the surface damage of brick walls caused by different regional microclimate environments is also different. When the inspectors are inexperienced, the damage assessment will be inaccurate, which will have a serious impact on the later structural safety assessment and repair. Later, data analysis and processing require a lot of experienced professionals, which will consume a lot of energy and time. Therefore, traditional professional human-based detection methods are difficult to meet the needs of rapid damage detection in historical buildings. With the development of scientific testing instruments, sensor-based structural damage detection technology [5,6], sensitivity-based model dynamic monitoring technology [7], combined experiment and numerical method detection [8,9], and other technical means have gradually emerged. However, the use of scientific equipment and instruments necessitates the use of professionals. If installed in a large historical building, the cost is high and the operation is more difficult. In recent years, scholars have carried out many studies on vision-based structural health monitoring, which are gradually replacing traditional manual inspection and widely used in the inspection of concrete pavement and bridge structures [10,11]. With the development of computer vision image technology, deep learning has been further studied in detection technology: for example, to apply CNN (Convolution Neural Network) and 3D modeling technology to detect concrete defects [12]; use a deep convolutional neural network (DCNN) to train a model for automatic pavement crack detection [13]; and use a faster R-CNN to detect concrete, steel, and the damage of the bolts [14]. The historical buildings built with gray bricks are unique, and the detection technology that was only applied to modern civil engineering of bridges and pavements in the past is difficult to be compatible with. The computer vision image technology applied to the identification, detection, and classification of historical gray-brick buildings still needs to be further explored.

1.3. Problem Statement and Objectives

As mentioned above, Macao is affected by complex and diverse climates. The Macau peninsula has undergone more than one hundred years of reclamation and land reclamation changes, and the foundations of many historical buildings are more complicated. What is even more surprising is that due to the influence of climate factors such as typhoons, flooding, and internal waterlogging, more types of damage have appeared on the surface of the gray-brick historic buildings. Because the gray bricks are manufactured using anoxic cooling, the operation is more troublesome, so now the production is relatively small, and there are fewer and fewer experienced restoration craftsmen. In order to improve the identification of the damage to the historical brick buildings and better estimate the cost of subsequent repairs, this paper proposes to use the YOLOv4 model in machine learning to automatically detect five types of damage to the historical brick buildings: missing, cracking, P&M (plants and microorganisms), yellowed, and stains. In this paper, the researchers explored the following four questions:

(1)

In the process of collecting data from on-site investigations and photographs, how many types of damage can the gray bricks be divided into?

(2)

How does machine learning help build the core technology that helps find different kinds of damage to bricks?

(3)

How effective is the model for training machines? How accurate is automatic detection compared to manual identification?

(4)

What is the result of the image recognition and analysis of the damage type of the gray brick?

2. Gray-Brick Walls and Climate Influence Factors

2.1. Material Characterization in the Gray-Brick Ancient Buildings

Gray bricks are calcined during the brickmaking process. The fuel in the brick kiln is burned in the absence of sufficient air. Most of the iron elements in the bricks are converted into ferrous iron, and the fired bricks are blue-gray. The density is 2.0 × 10³ kg/m³ to 2.5 × 10³ kg/m³, and the compressive strength is greater than 10 MPa. The building parts involved in brick work mainly include brick walls, brick arches, and brick eaves. The positions of these parts in the building are shown as follows (Figure 2). Such a masonry method is commonly used on the door openings connecting spaces in brick arches. Gray bricks, for example, are frequently used in these locations in Macau’s historical residential buildings.

There are several common specifications and sizes of gray bricks: 60 × 240 × 10 mm, 75 × 300 × 120 mm, 100 × 380 mm, 100 × 400 × 120 mm, 200 × 400 mm, 240 × 115 × 53 mm, 400 × 400 × 50 mm. The traditional gray-brick walls can be divided into three categories: (1) silk seam walls, which are more beautiful and are mostly used for the outer corners of main walls such as building facades; the joint width is not greater than 1 mm. (2) Grinding wall, mostly used for main external walls such as gables; joint width 4–8 mm. (3) Rough masonry walls, or walls with lower grades, are mostly used for interior walls and unimportant walls; the joint width is 8–10 mm (Figure 3). In this study, the images of the gray bricks were mostly taken from the outside corners of the important facade walls of traditional Macau buildings.

2.2. Analysis of Damage Types and Climatic Factors of Gray bricks

Through the investigation of the gray-brick ancient buildings in the whole area of Macau, the damage to the gray-brick ancient buildings is mainly divided into five types, namely: brick missing, brick cracking, bricks attached by plants and microbes, bricks turning yellow, and bricks with stains (Figure 4). In addition, in order to make the effect of target detection and recognition clearer in the future, the above five kinds of brick damage conditions are named with tag names.

Brick damage is closely related to climatic conditions. The damage to the gray bricks caused by different climatic conditions is also different (

Table 1. The relationship between the types of damage to gray bricks and climate factors.

No.	Type of Damage	Damage Degree (From Big to Small)	Climatic Factors
1	Brick missing	(1) The whole brick body is missing. (2) The bond between brick and brick is missing. (3) Part of the surface of the brick body is missing and falling off.	(1) Excessive humidity will make the brick wall damp, which will cause corrosion and loss of the brick wall surface. (2) Excessive wind force will cause the brick wall to be exposed to wind and rain, resulting in damage and loss of the brick wall surface. (3) Acid rain will dissolve the alkaline substances on the surface of the gray bricks, making the gray bricks weak and breakable.
2	Brick cracking	(1) The overall longitudinal cracking of the brick body. (2) Partial cracking of the brick body. (3) There are slight cracks in the brick body.	(1) The humid environment will erode the brick wall when the humidity changes, and cracks will appear.
3	Bricks are attached by plants and microbes	(1) Plants and microorganisms attach to the brick body, and the brick body sprouts leaves. (2) The brick body is thicker than 1 mm and has a connection with plants and microorganisms. (3) The brick body has obvious color changes and microorganisms attached.	(1) Humid climates can make brick walls susceptible to plant erosion because moisture on brick walls absorbs plant roots, making them more likely to erode brick walls.
4	Bricks turn yellow	(1) The whole brick body turns yellow. (2) Part of the brick body turns yellow. (3) The brick body is faintly yellowish.	(1) Brick walls are vulnerable to oxides in the air in humid environments, and these oxides make the brick walls yellow. (2) A humid climate will make the brick wall’s sealing layer useless, which will cause the water on the surface to evaporate and turn the bricks yellow.
5	Brick with stains	(1) There are stains attached to the brick body as a whole. (2) Stains are attached to the brick body. (3) Stains are attached to the joints of the bricks.	(1) In the morning, there are lower temperatures, and due to long-wave radiation, the wall surface is subjected to undercooling, in turn leading to surface condensation. At that moment, particulate matter in the air adheres to the moist surface. Throughout the day, the moisture evaporates, leaving the particulate matter attached to the wall.

). Since Macau has a subtropical maritime climate, the factors affecting the climate are mainly humidity, temperature, wind, and acid rain; they are not affected by frost. Elevated humidity will increase the water content on the surface of the gray brick, thereby increasing the possibility of seepage and condensation, which in turn will aggravate the corrosion of the gray brick. An increase in temperature will lead to an increase in the temperature of the surface of the gray brick, thereby accelerating the evaporation of water and possibly causing local thermal cracking. Weathering refers to the fact that, under the action of the wind, dust, sulfate, and oxide substances in the air form a thin film on the surface of gray brick, thereby accelerating the corrosion of that surface.

Table 2. Types of gray-brick damage and common repair methods.

No.	Type of Damage	Common Repair Methods
1	Brick missing	(1) Brick repair: Use new bricks to fill the missing bricks, and then use cement powder and mortar to bond them; the surface of the repaired bricks should be consistent with the original surface.
2	Brick cracking	(1) Concrete repair: fill the cracks with concrete, smooth the concrete with a trowel, then coat the surface with cement powder to match the original brick. (2) Broken brick repair: Put the broken bricks in the cracks, fix the bricks with mortar, then smooth the surface with a trowel to match the original facing bricks. (3) Mortar repair: fill the cracks with mortar and smooth the surface with a trowel to match the original brick. (4) Reinforcement tie: Put the reinforcement into the crack, fix the reinforcement with mortar, then smooth the surface with a trowel to make it match the original brick.
3	Bricks are attached by plants and microbes	(1) Algae removal: Use algicides (such as ammonium sulfate or sodium chloride) to remove plants and microorganisms from the surface of the gray bricks. (2) Cleaning: Use cleaning agents (such as sodium carbonate or acetic acid) to remove plant and microbial residues from the surface of the gray bricks. (3) Renovation: Use refurbishment agents (such as cement powder or lime powder) to remove plant and microbial residues from the surface of the gray bricks. (4) Coatings: Use coatings (such as cement coatings and epoxy coatings) to remove plant and microbial residues from the surface of the gray bricks.
4	Bricks turn yellow	(1) Cleaning: Clean the surface of the gray bricks with water and detergent to remove dirt and stains from the surface. (2) Coating: Put a layer of gray lime powder on the surface of the gray bricks to restore its original color and gloss.
5	Brick with stains	(1) Scrub with warm water: Use warm water to scrub the stain, then wipe it dry with a damp cloth. (2) Detergent scrubbing: Use a special stain cleaner (such as washing powder or detergent), apply it evenly on the stain, then wipe it with a damp cloth to remove the stain. (3) Polishing: Use a polishing machine to clean off surface stains, oil stains, etc.

lists the repair methods for the damaged bricks. The repair method of the damaged brick is of great significance for the study of the damaged type of the gray brick. On the one hand, repair methods for damaged bricks can help researchers better understand the types of brick damage. Distinguish the difference between the damage conditions of the blue bricks, so as to better confirm the type of damage and the detection target. On the other hand, there are economical differences among the repair methods of different blue brick damage. The repair costs of different damage conditions are quite different, and specific differences are required. In order to evaluate the repair cost more quickly, it also reflects the value of setting multiple detection labels in this study.

The different damages to gray bricks require targeted repair methods (Table 2). Structural damage can be repaired with repair mortar or cement powder to ensure the structural integrity of the brick. For surface damage, paint can be used to repair the damage on the brick surface so that the brick surface becomes smooth and beautiful again. For bricks that are seriously damaged, the old bricks can be demolished and replaced with new bricks to ensure the integrity of the bricks. In addition, polishing can also be used to clean off surface stains and oil stains through a polishing machine so that the brick surface becomes smooth and beautiful again. In short, different types of brick damage can be repaired accordingly, so that the brick can become complete again and restore its original appearance.

As mentioned above, gray-brick materials have various damage types and repair methods, and it is necessary to scientifically identify the damage types and adopt corresponding repair methods. This is the basic and most important work in the restoration management of ancient buildings, and the cost of restoration can be estimated in advance. In the past, the quality inspection of gray bricks usually used manual methods, which required a lot of manpower and time and had low efficiency. Because of the subjective factors of the testers, it is difficult to guarantee the accuracy and reliability of the test. However, in the past research on using deep learning to detect the quality of bricks, only one or two types of brick damage were studied, and most types of brick damage could not be covered [15,16]. The detailed identification and detection of gray-brick quality can be used to guide the selection of subsequent material restoration methods.

Therefore, this study screened out five main types of gray-brick damage based on the current situation of ancient gray-brick buildings in Macau. These five types cover the majority of Macau’s damaged bricks. Using the method of target detection in machine learning, the automatic detection of the quality of gray bricks can be realized more comprehensively and efficiently.

2.3. Sample Processing

In order to train the model of machine learning, it is necessary to collect and process data as the material for machine learning. In this study, the researchers photographed and recorded most of the ancient gray-brick buildings in Macau and took 1355 pictures of gray bricks. These photos are mainly of the facades and walls of ancient gray-brick buildings, covering as much as possible the current situation of the gray-brick buildings in Macau (Figure 5). Except for the Mandarin’s House in the historic center of Macau, the locations of the photo collection are all located in the World Heritage Buffer Zone. The main ones are: Pátio da Iluso, Patio das Seis Casa, Patio do Sal, and the historic buildings with gray-brick walls left in the nearby streets.

After manual review of all the pictures, 1000 pictures with good quality and certain representativeness were selected as material samples for machine learning. To improve machine learning efficiency and standardize materials, all photos were further processed to 512 × 512 pixels in size. Finally, the researchers marked the types of brick damage one by one on the 1000 photos. After that, another group of researchers reviews and outputs the corresponding photo and label files.

3. Methods

3.1. Operational Process of Image Recognition Technology

This paper proposes the method of using machine learning to realize the automatic detection of the quality of gray-brick walls (Appendix A). It provides a scientific method for the daily maintenance and repair of ancient gray-brick buildings. Therefore, it provides important practical value for the batch detection of large-scale and multi-building gray-brick ancient building settlements. The research method mainly consists of seven aspects: data collection, data unification, data labeling, YOLOv4 model training, model testing, result analysis, and model application (Figure 6). Data collection is mainly reflected in the researchers’ collection, sorting, slicing, and statistics of gray brick photos. In the study, the main part collected is the most easily accessible part of the historic building, that is, the most important external wall envelope. For example, the indoor kitchen and other parts of the roof are not entirely made of gray bricks, so they are not within the scope of this research.

First of all, as mentioned above, through early data collection and unified processing, the data are manually labeled. The reason for labeling the pictures is to provide learning samples for the machine for the next step of model training and to train the machine to recognize different situations of gray-brick damage. In this study, the researchers were divided into two groups. The first group of researchers had to label 1000 images of identical gray bricks one by one. The contents of the labels were determined according to the damage types of the above five kinds of gray bricks. Another group of personnel needs to check the labels processed by the previous batch of personnel to ensure the accuracy of the data. In addition, for pictures that are prone to ambiguity, such as the “stains” of gray bricks and the “infestation of plants and microorganisms,” it is often not easy to distinguish from the pictures, so repeated on-site inspections and judgments by cultural relic experts are required.

Second, for training, the object detection model YOLO v4 in machine learning requires paired data of pictures and labels. The process of model training is long, and it is necessary to always pay attention to the LOSS value of each generation of training results. Adjust the training parameters in time to achieve the best training results based on the trend of the LOSS value. The loss value is an indicator used in machine learning to measure the gap between the predicted results of the model and the actual results, and it can reflect the degree of fitting of the model. Its function is to evaluate the performance of the model and it can be used to guide the model’s training. Through continuous iterative training, the loss value becomes smaller and smaller, so as to achieve the purpose of improving the performance of the model.

Finally, after the training is completed, a batch of weight files will be generated. The weight file is made up of the parameters learned during the model training process in machine learning, and they represent the knowledge learned by the model during the training process. At the same time, these weight files also represent the training results for different LOSS values. Because the LOSS value cannot fully represent the accuracy and reliability of gray-brick recognition, Therefore, these files need to be tested. In this study, a new batch of gray-brick images is prepared for testing. Use different weight files to detect the same batch of gray-brick pictures, and finally, adopt the weight file with the best detection effect as the actual application.

3.2. Model Training

The model is based on YOLO v4 [17], which has excellent recognition accuracy and efficiency. Using CSPDarkNet53 as the backbone feature extraction network, this network contains 72 convolutional layers, which significantly improves the overall performance. Its network structure has three characteristics: (1) The backbone feature extraction network is CSPDarkNet53. It promotes the fusion of underlying information and enhances feature extraction capabilities; (2) spatial pyramid pooling adopts SPP [18] and performs maximum pooling operations on four different scales in the output of the last layer. It effectively extracts the most significant contextual features; (3) on the basis of the structure of the feature pyramid network FPNet [19], it is further improved into a path aggregation network PANet [20]. Add a top-down structure to FPNet’s bottom-up structure to extract and fuse features at different scales.

The network framework of YOLO v4 is shown in Figure 7. Spatial pyramid pooling (SPP) is located at the junction of the backbone network and the neck network, which converts the input feature map into feature maps of different sizes through maximum pooling. Then, the feature maps of different sizes (5 × 5, 9 × 9, 13 × 13) are combined with the original feature map for connection operation. As a new feature map, this method can better increase the receptive field of the convolution kernel. The feature pyramid network (FPNet) and path aggregation network (PANet) are used in the neck network part of YOLO v4. FPN connects the high-level feature map with the low-level feature map through upsampling operations, increasing the amount of information in the feature map. PAN connects low-level features and high-level features through downsampling, which shortens the fusion path between layers and improves the extraction ability of network features.

The obtained features are used to predict in the YOLO head via the convolution, pooling, upsampling, and downsampling steps of the above network structures. The YOLO head has multiple scales (64 × 64, 32 × 32, 16 × 16), which solves the problem that small targets are difficult to predict. Each prediction header contains three layers. Layer 0 represents the position of the confidence in the detection result (red is the highest possibility, blue is the lowest). Layer 1 represents the label recognition of the detection results. Layer 2 is the weight of the first two layers, i.e., the low probability category (blue) in layer 1 is excluded on the basis of layer 0. Finally, the results of all prediction heads are combined, and a prediction result export picture is generated. The features obtained in this process will be saved in the weight file for later use.

In the first training, since the machine has no prior experience, there is no weight file related to the recognition of gray bricks. Therefore, using the transfer learning method, the backbone feature extraction network of the model is initialized with the pre-trained weights of the VOC dataset. The model is trained for a total of 200 epochs. Gradients are evaluated using the Adam optimizer, and update steps are calculated. Due to the limited number of training sets, the batch size is set to 2, and the learning rate is set to 0.001 for the first 10 times. Freeze the training of the backbone feature extraction network to speed up the convergence and avoid the destruction of the pre-trained weights. For the last 100 times, set the learning rate to 0.0001, unfreeze the backbone feature extraction network, and further train the entire model with a smaller initial learning rate, thereby speeding up the training time of the entire network.

4. Discussion: Image Recognition and Analysis of Damaged Gray Bricks

4.1. Model Test

The model training of each iteration cycle will generate the corresponding weight file. A weights file is a set of variables in a machine learning algorithm that describes the parameters of the model (such as the slope in linear regression). It is the result of model learning and can be worked out by the training algorithm. The weight file can reflect the parameters and structure of the model, so as to better understand the internal mechanisms of the model. Optimizing the model can also be used to evaluate the performance of the model and the changes in performance on different training sets. These weight files use two metrics to assess the training’s performance. Among them, “loss” refers to the loss function value of the model on the training set, which reflects the performance of the model on the training set, and val_loss refers to the loss function value of the model on the validation set, which reflects the performance of the model on the validation set. Loss value is an important concept in machine learning. It is the loss cost of the model during training, and it can also be called the error rate. The loss value can be used to measure the quality of the model. The smaller the model, the better the model, and the larger the model, the worse it is. The loss value is calculated by calculating the difference between the predicted value of the model and the actual value. Generally, a loss function, such as mean square error (MSE), cross entropy (Cross-Entropy), etc., is used to calculate the loss value. The physical meaning of the loss depends on the context of the training process. In general, lower values are better than higher values, so a value of 5 is better than a value of 100.

Figure 8 expresses different training times and their corresponding loss values. As shown in Figure 8, the loss value is higher during the first training (loss value is 34.861, val_loss value is 9.462). After dozens of iterations, the values of the two dropped rapidly to less than 7, and finally, remained flat between 5–6. Among them, at the 112th generation, the value of val_loss was the smallest (6.159); at the 138th generation, the value of loss was the smallest (5.816); and at the last iteration (200 generations), the values of loss and val_loss were 5.886 and 6.258, respectively. Overall, when the loss and val_loss values drop to 5–6, the model training has entered the bottleneck, and there is no further downward trend. The maximum number of iterations refers to the maximum number of iterations experienced in a training process in machine learning. The number of iterations is increased or decreased according to the needs of the experiment (the effect of model convergence). In this study, the number of iterations is set to 200, and the maximum number of iterations is the 200th epochs. Therefore, the weight files of epochs 112, 138, and 200 are selected as the objects of further testing.

Evaluate model performance to compare the performance of different models in order to find the best model. First, the weight files for the above three different parameters are used to control the output results of the model, respectively. The samples for the test are randomly selected from 1000 training set samples, named A to F, and different weight files are loaded into the model to output the results. Figure 9 shows that the representative (minimum Loss value, Val_Loss value, and maximum number of iterations) weight file corresponding to the Loss value is extracted from Figure 8 for model testing. It can be seen that the detection effect of the model corresponding to the minimum loss value is the best (the 138th generation of model training). If it only trains five times, the best detection effect will not be achieved.

The confidence level of 0.5 refers to the machine learning algorithm’s confidence level for a specific result; that is, the machine learning algorithm believes that the probability of a specific result is 0.5, which is 50%. The three columns on the right side of Figure 10 express the impact of setting different confidence levels on the detection results of gray bricks. As shown in Figure 10, under the same confidence level (0.5), the following findings are found: (1) Max Epoch has the best detection effect in sample E and can identify more gray-brick stains. (2) The detection effects of Min Val_loss and Max Epoch are similar, showing the same detection results in samples B, C, and F, and all models have consistent detection results in sample B. (3) Min Loss has the best detection effect in samples A, C, D, and F and can detect more cracks and stains in gray bricks. Overall, the effect of the Min Loss model is relatively better, but the detection results are not comprehensive, so try to further reduce the confidence. Confidence is a parameter to measure the accuracy of the model. The higher the confidence, the higher the reliability of the detection results. In this study, by reducing the confidence (0.3 and 0.1), the model can detect more brick damage (Figure 9). The recognition results have been manually verified, and most of them conform to the actual situation of brick damage, and the reliability of the detection results remains within an acceptable range. Weigh the quantity and quality of the gray-brick detection results, finally select the weight file of Min Loss, and set the confidence level of 0.1 as the model and parameters for further testing.

In order to further understand the internal working principle of the model, the head and layer are converted into pictures so as to better observe the model parameters and test the detection accuracy. Using the above model, test the new sample picture and output the 3 layers of 3 heads in YOLOv4 to observe the effect of feature extraction. The head is the prediction layer in the model, which is responsible for extracting the features of the image. YOLOv4 has three heads, each with its own detection grid. They detect objects of different sizes and can detect smaller objects more accurately. In this study, the grid sizes of these 3 heads are 64 × 64, 32 × 32, and 16 × 16, respectively.

As shown in Figure 11, layer0 is the score of confidence, layer1 is the vector of labels, and layer2 is the result of combining layer0 and layer1. Specifically, layer0 can capture the specific location of the damaged brick in the test picture (the higher the confidence level, the redder the color, and vice versa). Layer1 marks each position of the picture (the higher the position that can be marked, the redder the color, and vice versa). Finally, layer2 combines the size of the confidence level and the position of the label to output the result, and the final detector will draw the bounding box coordinates (center, height, and width) according to the result of layer2.

Through the above process, some characteristics of the test sample and model can be observed: (1) The feature extraction of the test image mainly occurs in head0. This is because the shooting distance of the test picture is relatively short and the scale of the gray-brick is relatively large, which belongs to the suitable grid scale of head0. (2) Head0 can detect the gray bricks that are relatively complete in the picture, and head1 and head2 can detect the gray bricks that appear on the edge or in part of the picture. (3) The detection reaction in layer1 is more intense, and the degree of reaction increases step by step from head0 to head2. This is because the main function of layer1 is to classify the labels of the gray bricks in the picture, and the scale of the detection grid from head0 to head2 gradually decreases. Therefore, the density of the classification increases correspondingly, showing a step-by-step response status.

4.2. Model Application

First of all, from the gray-brick ancient buildings in Macau, some photos were retaken as materials for the model application. The photos have not undergone any post-processing and are 6000 × 4000 pixels in size. At the same time, there is an offset in the shooting angle of the handheld camera so as to verify whether the gray-brick detection will fail due to the scale and angle of the photo in general applications. Figure 11 shows the feature extraction effect of the layer in the detection process of one of the above photos. The following findings were found in the research: (1) The model is compatible with the detection of gray bricks of different scales. Compared with the pictures in the training set, the shooting range of the photos is larger, and the number of gray bricks is higher. Each head has played a specific role, particularly head2 with its small detection grid (16 × 16). (2) The model can eliminate the interference in the photo during the process of detecting gray bricks. There are wires and other facade materials on the right side of the photo, and in layer1, there is a more intense reaction on the right side, but it was not misidentified in the final detection results. (3) The model is not affected by the shooting angle. The photos taken on the spot have a certain overlooking, but the final detection effect is still ideal.

Figure 12 shows the pictures taken at different gray-brick ancient building projects in Macau and the results of their detection. It can be seen that the gray-brick detection effect of the model is good overall, and most of the damaged bricks are identified, but there are certain errors in special cases. and found the following: (1) When the gray-brick lacks a significant outline, the model will make mistakes. In the upper right corner of project 1, a large area of gray bricks is covered by cement, and the outline of the gray bricks is difficult to recognize. It is recognized by the model as the lack of gray bricks, but it should actually be stains caused by cement covering. (2) In the case of rough bricks, the model cannot accurately distinguish the difference between missing bricks and stains. In project 2, due to the fact that there are many fine scratches on the brick wall as a whole and at the same time under high-definition photos, the model is more sensitive to the rough marks on the surface of the gray bricks, and the slight scratches on the gray bricks are mistaken for missing. (3) Materials similar in color to gray bricks may be mistakenly identified as gray bricks. The lower right corner of project 3 is not a gray brick, but it is detected by the model as a plant and microbial attack (p&m). (4) Extremely oblique angles will cause the model to fail to recognize The right side of project 4 is the corner of the brick wall, and the gray bricks in this part have not been identified.

As mentioned above, the model in this study can effectively detect the type of brick damage. However, there is still room for improvement in the detection accuracy, and it still cannot completely replace professionals to do more accurate detection, although it can be used as an auxiliary tool for gray-brick detection. In particular, the cost of repair craftsmen for traditional buildings is relatively high, and when it takes a long time to hire repairs, this method can be used to preliminary estimate the damage to gray-brick historical buildings and the corresponding range of maintenance costs. In addition, although all the materials in this study come from Macau, there are still many other regions and countries in the world that use gray-brick materials to build buildings or structures. In the future, the research can be further expanded to other regions and countries.

4.3. Manual Validation of Models

In order to verify the reliability of the model, 1000 gray brick samples are used to verify the model. The inspectors are composed of a cultural relic expert and two architecture scholars. They manually checked the test results of the model one by one and found that among the 1000 test results of the model, there were 857 valid test samples, 136 wrong test samples, and 7 missed test samples (specific numbering details are in Appendix B). Therefore, the following conclusions can be drawn: (1) The results of this experiment show that the detection effect of the model is good, and 85.7% of the samples can be detected effectively, but 13.6% of the samples are still detected incorrectly, and 0.7% of the samples are not detected (Figure 13). (2) This result may be due to an imbalance in the sample data, i.e., the number of samples for some damage cases is too small, making it difficult for the model to learn such damage cases, or the training data of the model are not sufficient, making it difficult for the model to generalize to all damage situations; or maybe the structure of the model is not complex enough to capture all the damage details. (3) In order to improve the detection effect of the model, further research in the future needs to continue to optimize the model, increase the number and diversity of training data, or replace a more complex model structure. In general, this study shows how machine learning can be used to find cracks in brick walls. It also shows the mistakes and flaws in the model, which needs to be constantly improved and perfected.

5. Conclusions

Macau belongs to the subtropical maritime climate area, and the hot and humid seasons easily lead to the aging and damage of building materials. Among them, the damage to the ancient gray-brick buildings that are common in Macau under the influence of the climate is particularly obvious. Therefore, relevant personnel need to conduct regular inspections on ancient brick buildings and take corresponding maintenance measures.

Due to the large number of gray-brick ancient buildings in Macau, the work of manual inspection one by one is cumbersome. Therefore, this paper proposes an automatic detection method for the damage type of ancient brick buildings based on machine learning. This method is based on the target detection YOLOv4 framework and is used for the detection of five types of gray-brick damage: brick missing, brick cracking, bricks attached by plants and microbes, bricks turning yellow, and bricks with stains. The model was trained for 200 iteration cycles using 1000 pieces of material collected on site and then tested on three different weight files, finally showing an overall good detection effect in practical applications. Compared with manual detection, this method is more efficient in the detection of large-scale gray-brick ancient buildings, which provides an important auxiliary role for manual detection.

The value of this study is mainly reflected in three aspects: (1) It can help government departments and historical building protection associations understand the state of historical buildings more accurately, so as to better protect these buildings. (2) This information can also help architects and engineers better plan restoration work and ensure that the restoration process does not damage the structure or appearance of historic buildings. (3) This information can also help academic researchers better understand the construction methods and techniques of historical buildings and provide an important basis for the protection and research of historical and cultural heritage.

All materials in this study were photographed by the researchers in Macau, and the field application of the model proves that the proposed method is effective for gray-brick damage detection. However, there is still room for improvement in the model’s detection accuracy, and the model can be improved in three aspects in the future. (1) Increase training data: collect more labeled training data so that the model can better fit the data, thereby improving recognition accuracy. (2) Data enhancement: Using data enhancement techniques, such as rotation, scaling, cropping, etc., can increase the generalization ability of the model and improve the recognition accuracy. (3) Adjusting parameters: Adjusting the parameters of the model, such as learning rate and activation function, can adjust the performance of the model.