Machine Learning Methods in Damage Prediction of Masonry Development Exposed to the Industrial Environment of Mines

Abstract

This paper presents the results of comparative studies on the implementation of machine learning methods in the damage intensity assessment of masonry buildings. The research was performed on existing residential buildings, subjected to negative impacts of the industrial environment induced by coal mining plants during their whole technical life cycle. The research was justified on the grounds of safety of use, as well as potential energy losses and CO₂ emissions generated by the inefficient management of building materials resources resulting from poor planning of retrofitting. In this field, the research is in line with the global trends of large-scale retrofitting of existing buildings in European countries due to their thermal insulation parameters and seismic hazard. By combining this with the effects of material degradation throughout the technical lifecycle of buildings, the proposed methods allow for a more efficient approach to maintaining quality management of large groups of buildings, which is part of the sustainable development framework. Due to the multidimensionality of the undertaken problem and the necessity of mathematical representation of uncertainty, it was decided to implement a machine learning approach. The effectiveness of the following methods was analysed: probabilistic neural network, support vector machine, naive Bayes classification and Bayesian belief networks. The complexity of individual methods dictated the order of the adopted research horizon. Within such a research plan, both model parameters were learned, and model structure was extracted from the data, which was applied only to the approach based on Bayesian networks. The results of the conducted analyses were verified by assuming classification accuracy measures. Thus, a method was extracted that allows for the best realisation of the set research objective, which was to create a classification system to assess the intensity of damage to masonry buildings. The paper also presents in detail the characteristics of the described buildings, which were used as input variables, and assesses the effectiveness of the obtained results in terms of utilisation in practice.

1. Introduction

Damage to buildings is generally considered according to two criteria: safety and serviceability. They constitute the basis for the design and are verified during the technical life cycle of the building’s structures. However, currently, along with the development of the idea of energy-saving construction and the reduction of CO₂ emissions in the atmosphere, the scope of requirements for building structures has been extended to include energy demand and durability requirements. The above aspects are included in the idea of sustainable development in construction [1,2]. Currently, the result of such activities is the ongoing transformation of the previously applicable safety and serviceability criteria, including socio-economic and environmental aspects [3]. Therefore, in the analysis of the damage process, the number of necessary factors to consider also increases. Some are also characterised by a large degree of uncertainty, which is most often described in probabilistic notation [4]. This, in turn, enforces the use of complex tools, such as multiple-criteria decision-making (MCDM) systems in the design process [5,6].

When considering the failure phenomenon from the point of view of structural mechanics, this process is initiated when the permissible threshold of the potential elastic deformation energy stored in a given structural element is exceeded [7,8]. It is the main criterion in the design of building structures and determines the maintenance of safety and acceptable levels of serviceability [9]. In turn, looking through the prism of the current requirements in the field of energy characteristics of buildings, damage often contributes to the development of leaks and moisture on the wall surfaces. They affect the deterioration of the thermal insulation properties of building partitions (walls and roofs) and generate increased demand for thermal energy needed to dry moisture and heat rooms. In this context, damage manifests itself more as being detrimental to serviceability, including the energy characteristics of buildings. Therefore, the correct prediction of the intensity of damage, especially in the case of existing building structures, may turn out to be crucial in terms of effectively preventing the loss of thermal properties, especially in the case of energy-efficient construction [10] or passive [11].

In the described case of a large group of masonry buildings located in a mining area, we deal with both threats, i.e., safety hazards and deterioration of performance characteristics affecting serviceability [12]. This is because these buildings are subjected to temporary or long-term impacts from the ground throughout their technical life cycle. Transient impacts include mining tremors, which result from a disturbance in the equilibrium of the rock mass, which leads to the release of a large amount of stored potential energy. This effect manifests itself at the surface in the form of ground vibrations and constitutes a kinematic loading for buildings located on the affected area [13]. In turn, the long-term impact results from the influence of large-scale subsidence as a direct result of underground mining exploitation or indirectly related hydrogeological disturbances [14].

Both of these effects constitute an additional load on the structure of building objects and often cause an increase in the stress of their load-bearing elements, which results in the phenomenon of damage.

Considering that the coal mining industry still operating in some countries is a pillar of the energy sector, it can be assumed that this method of obtaining energy, in addition to environmental pollution, also generates negative effects in the form of damage, which should be considered in terms of safety, socio-economic problems and environmental.

The last issue, which is not without significance for the global trend of reducing energy expenditure and releasing CO₂ into the atmosphere, is the production of building materials [15,16]. Unfortunately, ineffective predictions of the range and intensity of damage at the stage of determining the extent of retrofit or repair work required, especially in existing buildings, contribute to the waste of building materials already at the stage of their production [17]. Given the large number of buildings at risk of damage, the overestimation is so large that they can cause a significant loss of irreversible energy and contribute to a further increase in CO₂ emissions (Figure 1).

When assessing the damage in individual masonry buildings, numerical methods, such as FEM [18,19], DEM [20,21] or FDEM [22,23] can be used. Unfortunately, when the damage risk assessment of a large number of buildings is required, the numerical computational approach becomes ineffective. What is more, the attempt to model such a large set of structures is almost impossible from a practical point of view.

Therefore, advanced statistical methods dedicated to analysing large datasets are applicable in such cases. This group includes methods from the field of artificial intelligence (AI), in particular, machine learning (ML). Data for such studies are acquired in situ and contain all the necessary information to map the process of damage in building structures.

A similar strategy is also very often adopted to assess the risk of buildings in seismic areas [24] or areas subject to the risk of floods [25], tsunamis [26] or tornadoes [27].

This paper presents the results of analyses carried out on a wide group of buildings subjected to the influence of the industrial environment of mines. Regarding the assessment of this phenomenon, the problem addressed in this paper shows several analogies to the impact of an exceptional natural environment on buildings.

2. Characteristics of the Mining Impact on Building Structures in the Context of Damage Occurrence

Synthetically, this chapter presents fundamental information on the negative consequences of mining impacts on building structures, particularly on the risk of damage resulting from such influences.

The implementation of the assumed research goals required collecting data on the impacts of the mining industrial environment. During the passage of underground mining exploitation, deformation occurs on the ground surface. In order to relate terrain deformation with the problem of surface development risk, detailed measurements of horizontal and vertical displacements were made. These measures are inclinations (T), curvatures (K) and horizontal deformations (ε) of terrain. As a result of geodetic measurements [28] or on the basis of model studies, it is possible to determine the values of the above-mentioned parameters.

Figure 2 is presented and interpreted schematically as the process of creating a mining basin.

The behaviour of the building in any position on the basin is shown in Figure 3. As a result of soil deformation, the building moves from its initial 1–2–3–4 position to the 1′–2′–3′–4′ position. In this process, when treating the object as a solid block, there is a vertical lowering (w_b) and a horizontal shift (u_b) of the building S’s geometric centre, and the building’s rotation, determined by its deflection T_b. Additional settlement of Δs_b may also occur, resulting from the horizontal loosening of the soil and causing the building to assume the position of 1′′–2′′–3′′–4′′ [24] finally.

In this position, two situations can be distinguished: the convex or concave rim of the mining basin.

Figure 4 shows the schematic damages in buildings located on the convex and concave edges of the mining basin. Examples of real mining damages will be shown in Section 4.

From 2011 to 2017, the analysed residential buildings were subject to the influence of coal mining exploitation.

Table 1. Characteristic parameters of mining operations in the analysed area.

Deck	Wall	Height [m]	Depth [m]	Period of Exploitation
503	4	2.6–3.3	625–720	2011–2013
510	30a and 31a	2.0–2.4	725–805	2013–2015
503	5 and 6	2.0–2.3	670–680	2015–2017

presents the characteristic parameters of the conducted mining operation.

Building a database appropriate for the analysis requires collecting information on mining influences in the locations of individual buildings. For this purpose, information on the forecasted values of horizontal soil deformation (ε-cf. Figure 2) was collected. The values of the mining influences result from mining forecasts regularly verified and approved by the results obtained on the geodesic measuring lines run along streets and at scattered points located on buildings. Figure 5 presents the quantity of buildings affected by the influence of horizontal ground deformations, with an accuracy of 0.5 mm/m. By using the influence of horizontal deformations, the resulting categories of the mining area were determined [30].

Due to the scattered and irregular values of horizontal deformations, it was decided to use the mining terrain category as a parameter defining the impact of mining operations on buildings.

3. Technical Specification of the Investigated Buildings

For basic analysis, a qualified group of 207 buildings was examined. During the field research, information about the buildings was collected and placed in a database.

Traditional multi-family residential buildings were erected in the period from the end of the 19th century to the 1940s. They were built in a compact, semi-compact development and as free-standing, fully or partially with a basement. The buildings were erected as two, three or four storeys.

Foundations are usually made of stone or brick. The walls in the basement storey level are made of stone or brick, while the walls of the upper storeys are usually brick. The basement ceilings are mostly ceramic on steel beams, less often in the form of vaults. Occasionally, the occurrence of a concrete floor on steel beams was observed. In the levels of the above-ground storeys, wooden ceilings, section ceilings in staircases and, occasionally, concrete ceilings on steel beams were made. The vast majority of door and window lintels are brick, with an arched or flat shape. The structure of stairs in the area of staircases is usually made of wood or steel, and less often, concrete.

Most of the analysed buildings were secured against the impact of mining exploitation during their use. Such reinforcement consisted of the buildings’ total or partial anchoring with steel bars in the ceiling levels.

Single-family buildings are low-rise buildings with a height of up to two above-ground storeys. In traditional brick technology, these buildings were erected until 2017, usually as free-standing or semi-compact buildings.

The construction of single-family houses is more varied and results from the construction period. Older buildings were built on stone and/or brick foundations, and newer ones on reinforced concrete foundations. Basement walls were made of stone or brick and later made of concrete blocks. The walls on the ground are made of brick or aerated concrete blocks, or ceramic blocks. Above the basement of older buildings, ceramic on steel beams, concrete on steel beams, or reinforced concrete were usually made. On the upper floors of the buildings, there is wooden, concrete on a steel beam, rib-and-slab concrete, or reinforced concrete ceilings. The ceilings of newer buildings are made of reinforced concrete or rib-and-slab concrete. Window and door lintels are mostly made from bricks, arched or flat in shape, and in some places, on steel flat bars or made from reinforced concrete in new buildings.

The mining protection of most of the older single-family houses was carried out in the phase of their use by applying an anchorage at the level of the ceilings. It is also possible to highlight some buildings, for which a perimeter reinforced concrete band was made at the level of the foundations. Newer structures were made with protection for the III or IV categories of the mining area with additionally reinforced concrete benches and ties, rib-and-slab concrete or concrete ceilings and ceilings with peripheral reinforced concrete rings.

Field research conducted for the 207 buildings concerned the designation of:

• Geometry: length, width, number of overground storeys, building area, volume, length of the building sequence, dilation method, the shape of the building, basement, variable level of foundation, variable building height;
• Construction as: foundation type, the material of basement wall, the material of the walls higher than the basement, mining influence securities, the ceiling above the basement, the ceiling above higher floors, lintels, additional data about anchoring;
• Other technical data: year of construction, repair factor, technical condition (natural wear), category of deformation resistance.

Below, in a graphical form in Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18, examples of distributions of the collected data are presented in the form of histograms (Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11) and bar graphs (Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18).

4. Damage Morphology in the Analysed Group of Buildings

Each building is classified into one of the four damage categories. These categories take into account the extent and intensity of damage and threats to the safety of the facility and its users [31].

The adopted building damage categories are described below:

• Possible damage in the form of slight scratches on the plaster of ceilings and walls, no structural damage.
• More intense damage to finishing and non-structural elements, i.e., scratching façade and internal wall plaster, trimming ceiling and wall plaster, scratching or local separation of soffits.
• Damages to structural elements; in the event of further deformation that can influence the extent, intensity and location of damages that may lead to the local loss of stability of structural elements or loss of bearing capacity.
• Damage that currently threatens the local bearing capacity of its elements; buildings with tremendous natural wear of structural elements.

Examples of damages for individual categories are presented in Figure 19.

Figure 20 shows the distribution of buildings damage categories from 2011 to 2017.

Finally, taking into account multiple damage checks for the selected 207 buildings, research material was collected containing 594 cases.

5. Characteristics of the Implemented Machine Learning Methods

Further activities were commenced with the database containing data on the area’s development (Section 3) and the impact of the industrial environment as mining operations (Section 2).

Bearing in mind the implementation of the assumed objectives of the study, further research focused on the classification methods because they allow obtaining prediction results in accordance with the adopted criterion of the intensity of damage to buildings described in Section 4.

An important advantage of the methods selected and described below is the possibility of adopting probabilistic notation at the stage of inference of a given system, as well as presenting the results of calculations in probabilistic notation, which can be interpreted as the risk of a given event.

By using the above information, a review of various machine learning methods was carried out, of which four were selected that can be used in the construction of the damage risk assessment model, and the methodology of which is presented below.

5.1. Probabilistic Neural Network

Neural networks are made up of individual computing units (neurons) that are connected in parallel. They are characterised by the presence of multiple inputs and one output. Neural networks can be used for both regression and classification tasks.

PNN is a specific network that learns to estimate the probability density function distributed over many so-called nuclei whose centres are represented by the data presented at the input [32].

Estimating the probability density functions (PDF) is based on the nuclear approximation [33,34]. Each of the analysed cases (observations) is located at a certain point in the input space, and a cluster of cases close to each other indicates an area of high probability density. On the other hand, regions distant from known cases are characterised by a probability density that goes down to zero. In nuclear estimation, simple functions (“kernel”) are located where each available case occurs, and then they are added together to obtain an estimate of the total probability density function. The parameter determining the shape of the probability density function is the smoothing parameter σ.

There are four layers in the PNN: input, pattern, summation and output [32]. The pattern layer consists of radial neurons that have parameters copied directly from the training data, and each corresponds to one case. Each of the radial neurons is modelled by a Gaussian function centred over the area bounded by the values of the variables of a specific training pattern. In the case of classification of each class in the summation layer, one neuron corresponds. Each of the summation neurons has connections from those radial neurons that have been positioned over the pattern centres of the training dataset belonging to that class. There are no connections between sum neurons and other classes of radial neurons. The output neuron sums the values appearing at the outputs of the sum neurons belonging to each class. Therefore, the output neuron value is proportional to the estimators of the nuclear probability density functions for different classes and may be an estimate of the probability of belonging to particular classes.

5.2. Support Vector Machine

The support vector machine was developed by Vapnik and has been in development since the 1970s [35,36]. It is a specific type of neural network that uses various activation functions and implements a learning method based on quadratic programming. SVM networks are used for both classification and regression tasks. Originally, this method was dedicated only to the problems of dichotomous (two-class) classification. Currently, it can also be used to obtain multiple classes when multiple classifications are carried out. The SVM method is also characterised by the fact that it can use various types of activation functions, including linear, polynomial, radial or sigmoidal functions [37].

According to the SVM method, the classification task is to find the optimal hyperplane that generates the widest margin of separation between patterns belonging to different classes [38]. Support vectors are very important; they are points in the data space that are the most difficult to classify and define the location of the separation hyperplane [39]. The obtained results are strongly influenced by the regularisation parameter C and the width of the functions of the γ nuclei. The higher the regularisation parameter C value, the narrower the margin of separation and the number of supporting vectors is reduced [40,41]. The γ parameter refers to the width of the assumed nucleus functions, and with its low values, the “range of influence” of the learning cases is greater [42].

The results obtained from using the SVM allow for the classification of new cases into one of the classes but do not provide information about the probability of this event. However, a method allows the transformation of a fixed lattice to probabilistic notation [43,44,45], in which the distance from the separation margins determines the probability values.

5.3. Naive Bayes Classification

In the naive Bayesian classifier, the algorithm assesses the probability of the occurrence of particular classes for the given input variables. The result is the prediction of the class that has the highest probability of occurrence of all classes.

NBC uses the Bayesian theorem concerning the conditional probability distribution. Using it requires much computational effort related to considering many conditional probabilities, which can be simplified by assuming that the input variables are independent of each other. Such an assumption of independence is often very optimistic and allows for a clear simplification of the calculations.

On the basis of the training dataset, two procedures are used to build the NBC classifier model:

• Maximum likelihood estimation (MLE), which maximises the conditional probability, is understood here as a verifiable thesis about the occurrence of individual classes for the training data;
• Maximum a posteriori estimation (MAP) maximises a posteriori probability of the occurrence of individual classes for the training set.

5.4. Bayesian Belief Network

The Bayesian belief network can be interpreted as an acyclic directed acyclic graph (DAG), which consists of nodes (variables) and the edges connecting them [46,47,48].

The graph structure encodes information about the interdependencies between the individual variables X = {X₁, …, X_N}. In terms of meaning, BBN represents the joint probability distribution over the set of random variables X [49].

For discrete variables, the model parameters ${\theta }_{X_j} = \left\{{\theta }_{ijk}\right\}$ are represented in the form of a multi-monomial conditional probability table (CPT). The total distribution $P\left(X\right|G,\Theta )$ is subject to decomposition on the basis of conditional local distributions $P\left(X_i\right|{\Pi }_{X_i},{\Theta }_{X_i})$. They are described above for each random variable X_i in relation to the corresponding set of conditioning variables (parents) ${\Pi }_{X_i}$. This formulation is possible thanks to Pearl [50,51] and his concept of conditional independence. Such activities enable a noticeable reduction in the number of relationships that do not show cause-effect relationships. It also makes it easier for the user to interpret the network’s structure.

The BBN learning procedure consists of two stages: structure learning and parameter learning [47,49,52].

There are three different approaches to learning BBN network structure from data: score-based structure learning, constraint-based structure learning, and hybrid algorithms [49,53,54]. The risk of damage to buildings can be determined using a number of variables with a seemingly small contribution. It has been found in the course of many studies described, among others [55,56,57]. Taking this into account and the characteristics of all described methods of learning BBN from data, it was decided to use the score-based approach in the further part of the research.

The second stage of creating a Bayesian network is learning the parameters, which result from the network shape determined at the structure learning stage. Generally, for a fixed network structure, the determination of θ results from the probability of the output variable occurring for the given input variables. This can be interpreted as counting the number of records for different conditions of the state combination of the parameterised vertex and its predecessors. These parameters are usually determined on the basis of the expectation maximisation algorithm (EM), which consists of determining the locally optimal estimator of the highest likelihood of parameters [58].

A very important additional advantage of the BBN method is the possibility of inferring in two directions and thus using the Bayesian network for diagnosis or prediction.

6. Results of Conducted Analyses

This chapter describes the methodology for creating the individual classifiers and presents a traditional way to determine the learning sets of the input data.

The measures used to evaluate the classification accuracy are presented and interpreted, which provided the foundation for the final comparison and selection of the most effective method.

In order to carry out the planned research, the database for analysis was first prepared. The next step was to build different models using four selected machine learning methods: PNN, SVM, NBC, BBN. From among the created models, on the basis of original criteria, the optimal model of damage risk assessment was selected.

The individual ML models were built in the R development environment [59]. The following packages were used to build the appropriate models: yap [60], e1071 [61], bnlearn [62], naivebayes [63], bnclassify [64], gRain [65,66] and caret [67].

6.1. Preparation of Analysis Data

In the process of analysing data preparation, all variables were discretised in terms of their further use in the learning process [68]. The categories of numerical variables were selected so that the number of cases in each category was not lower than 5% of the size of the entire set. Such a selection of categories influenced the preservation of the homogeneity of the database in terms of teaching individual models.

The dataset has been divided into training and test sets. The division was proposed in the proportion of 80:20. The stratified sampling approach was applied [69] to maintain the completeness of the patterns for the entire model building process. This allows for the retention of complete information, both for the learning process and for the later testing stage.

By keeping the above-mentioned procedures, the sets were separated, the number of which was, respectively, 478 cases in the training set and 116 cases in the test set. The proposed division was analogous to all the methods used in the research.

In accordance with the requirements for each method covered by the study, a training set was used only for learning, and a test set was used to evaluate the created models.

6.2. Applied Measures for Assessing the Classification Accuracy

The error matrix was used as a commonly used measure of classification correctness assessment to compare ML methods results.

Table 2. Confusion matrix for a binary classification.

	Actual Positive	Actual Negative
Predicted positive	True positives TP	False positives FP
Predicted negative	False negatives FN	True negatives TN

shows an example of a confusion matrix.

Overall accuracy is the essential comparative parameter [70] (1).

$$ACC = \frac{TP + TN}{TP + FP + FN + TN}$$

(1)

Additional parameters in the assessment were:

• Precision [70] (2):

$$PPV = \frac{TP}{TP + FP}$$

(2)

• Recall [70] (3):

$$TPR = \frac{TP}{TP + FN}$$

(3)

A very important feature in building models based on ML methods is the generalisation of knowledge obtained during learning. The relative difference in classification accuracy was calculated for the training set and test set (ΔACC), allowing for a reasonable comparison of the generalising abilities of individual models.

The obtained results and detailed descriptions are included in Section 6.3, Section 6.4, Section 6.5, Section 6.6 and Section 6.7. The division of data into training and test sets has been preserved. These matrices indicate the results concerning the classification accuracy and the average precision and recall, according to (1)–(3).

6.3. The Results for the PNN Method

For the construction of the PNN classifier, the yap package [60] in R was used, taking into account the standardisation of the variables. Standardisation of variables is the operation on their values, as a result of which the variable obtains the mean expected value of zero and the standard deviation equal to one.

It is very important for this method to determine the optimal width of the Gaussian functions of the nuclei, characterised by the parameter σ [32]. The optimal selection of this parameter was made by performing 4-fold cross-validation [71], and the optimisation method was the golden ratio method.

The obtained σ parameter value was 1.47. Optimising the value of this parameter made it possible to obtain the highest classification accuracy for the test set.

In accordance with the parameters specified in Section 6.2, the prepared model was assessed in terms of the classification correctness of the training and testing sets and its generalisation properties.

Table 3. Confusion matrix for the PNN classifier—Accuracy of classification, average precision and average recall for training and test sets.

Training Set Containing 478 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	37	1	0	0	38	97.37%
	2	4	252	15	0	271	92.99%
	3	0	0	138	5	143	96.50%
	4	0	0	0	26	26	100.00%
Σ		41	253	153	31	478	avg. PPV 96.72%
Recall TPR		90.24%	99.60%	90.20%	83.87%	avg. TPR 90.98%	ACC 94.77%
Test Set Containing 116 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	3	0	0	0	3	100.00%
	2	8	66	9	0	83	79.52%
	3	0	0	17	8	25	68.00%
	4	0	0	0	5	5	100.00%
Σ		11	66	26	13	116	avg. PPV 86.88%
Recall TPR		27.27%	100.00%	65.38%	38.46%	avg. TPR 57.78%	ACC 78.45%

presents the results.

The created PNN classifier shows a very good classification accuracy, which was 94.77% for the training set. The obtained results are satisfactory for the test set, with a classification accuracy of 78.45%. The generalisation capacity of the model is assessed as insufficient (ΔACC = 16.32%).

6.4. The Results for the SVM Method

The SVM classifier was built using the e1071 package [61] in R.

The SVM method with radial functions of the nuclei was used in the research. For the classification problem formulated in this way, the hyperparameters C and γ were determined by optimisation using the grid search method [72]. The e1071 package offers this possibility by implementing the tune.svm function [61]. The procedure of optimal selection of hyperparameters assumes 10-fold cross-validation. On the basis of the analysis performed, the optimal hyperparameters were designated with the following values: C = 10, γ = 0.1.

The SVM classifier, created for the extracted optimal values of the C and γ hyperparameters, was characterised by the number of supporting vectors equal to 390. The number of supporting vectors in relation to all patterns used during learning is therefore 81.80% of the size of the training set. This proves the high complexity of the model, which may translate into lowering its generalisation properties.

Table 4. Confusion matrix for the SVM classifier—accuracy of classification, average precision and average recall for training and test sets.

Training Set Containing 478 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	38	1	0	0	39	97.44%
	2	3	249	10	4	266	93.61%
	3	0	3	143	1	147	97.28%
	4	0	0	0	26	26	100.00%
Σ		41	253	153	31	478	avg. PPV 97.08%
Recall TPR		92.68%	98.42%	93.46%	83.87%	avg. TPR 92.11%	ACC 95.40%
Test Set Containing 116 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	4	0	0	0	4	100.00%
	2	7	59	10	1	77	76.62%
	3	0	7	16	7	30	53.33%
	4	0	0	0	5	5	100.00%
Σ		11	66	26	13	116	avg. PPV 82.49%
Recall TPR		36.36%	89.39%	61.54%	38.46%	avg. TPR 56.44%	ACC 72.41%

presents the results.

The created SVM classifier shows a very good classification accuracy, which was 95.40% for the training set. The obtained results are satisfactory for the test set, with a classification accuracy of 72.41%. Based on the classification accuracy results, the generalisation capacity of the model is assessed as insufficient (ΔACC = 22.99%).

6.5. The Results for the NBC Method

The NBC classifier was made with the use of four packages in the R software [61,63,64,73]. The best classification accuracy results were obtained for the classifier built using the naivebayes package [63]. The results presented below apply to this model.

The algorithm implemented in this package detects and assigns classes to individual variables [74]. The maximum likelihood method was used to determine the parameters for the CPT.

It was also necessary to use the Laplace smoothing parameter in order to build a valid NBC classifier. For lower values of this parameter, the classification accuracy increases. Unfortunately, along with the increase in its classification accuracy, its effectiveness in unusual cases noticeably deteriorates [12]. As a result of the multiple analyses carried out, it was found that good classification accuracy was obtained for the parameter equal to the value of pL = 10, and, equally important, the appropriate generalising properties were maintained.

Table 5. Confusion matrix for the NBC classifier—accuracy of classification, average precision and average recall for training and test sets.

Training Set Containing 478 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	28	33	12	0	73	38.36%
	2	12	216	1	8	237	91.14%
	3	1	3	140	6	150	93.33%
	4	0	1	0	17	18	94.44%
Σ		41	253	153	31	478	avg. PPV 79.32%
Recall TPR		68.29%	85.38%	91.50%	54.84%	avg. TPR 75.00%	ACC 83.89%
Test Set Containing 116 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	7	6	5	0	18	38.89%
	2	4	58	1	3	66	87.88%
	3	0	2	20	7	29	68.97%
	4	0	0	0	3	3	100.00%
Σ		11	66	26	13	116	avg. PPV 73.93%
Recall TPR		63.64%	87.88%	76.92%	23.08%	avg. TPR 62.88%	ACC 75.86%

presents the results.

The created NBC classifier shows a good classification accuracy, which was 83.89% for the training set. The obtained results are satisfactory for the test set, with a classification accuracy of 75.86%. The generalisation capacity of the model is assessed as satisfactory (ΔACC = 9.57%).

6.6. The Results for the BBN Method

In order to carry out the analyses in accordance with the BBN’s methodology, possible interactions of individual variables with each other were assumed.

The selected method of teaching classifiers has a significant impact on the network structure obtained with the use of BBN. Eight different methods of learning the network structure were selected, and their results were analysed. Accordingly, these methods are described in the bnclassify [64] and bnlearn [62] packages.

From among the analysed learning methods, Chow-Liu’s Tree Augmented Naive Bayes (TAN-CL) allowed us to obtain the best results. The selected method of training is the result of combining two simpler methods: the Tree Augmented Naive Bayes method (TAN) [75] with the Chow-Liu variable link detection algorithm [76].

The driving parameter in the model construction is the model fit measure that serves as a target for score-based optimisation. The influence of selected objective functions on the obtained results was analysed: AIC, BIC and log-likelihood (loglik); and the best results were obtained using the AIC criterion.

Table 6. Confusion matrix for the BBN classifier—accuracy of classification, average precision and average recall for training and test sets.

Training Set Containing 478 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	40	15	0	0	55	72.73%
	2	1	220	29	5	255	86.27%
	3	0	15	118	3	136	86.76%
	4	0	3	6	23	32	71.88%
Σ		41	253	153	31	478	avg. PPV 79.41%
Recall TPR		97.56%	86.96%	77.12%	74.19%	avg. TPR 83.96%	ACC 83.89%
Test Set Containing 116 Cases
Damage State Category after Impacts		Observed				Σ	Precision PPV
		1	2	3	4
Predicted	1	11	2	0	0	13	84.62%
	2	0	58	4	0	62	93.55%
	3	0	5	22	3	30	73.33%
	4	0	1	0	10	11	90.91%
Σ		11	66	26	13	116	avg. PPV 85.60%
Recall TPR		100.00%	87.88%	84.62%	76.92%	avg. TPR 87.35%	ACC 87.07%

presents the results.

The created BBN classifier shows a good classification accuracy, which was 83.89% for the training set. The obtained results are also good for the test set, with a classification accuracy of 87.07%. The generalisation capacity of the model is assessed as very good (ΔACC = 3.79%).

6.7. Comparison of Results Obtained by Different ML Methods

In order to compare all the results obtained with the use of machine learning methods, they were summarised in

Table 7. Comparisons of classification parameters for all used ML methods.

ML Method	ACC	avg. PPV	avg. TPR	ACC	avg. PPV	avg. TPR
	Training Set	Training Set	Training Set	Test Set	Test Set	Test Set
PNN	94.77	96.72	90.98	78.45	86.88	57.78
SVM	95.40	97.08	92.11	72.41	82.49	56.44
NBC e1071	63.18	57.42	65.97	60.32	55.24	58.71
NBC bnlearn	63.60	58.51	66.17	86.21	86.61	82.05
NBC naivebayes	83.99	78.43	75.51	76.72	74.55	64.80
NBC bnclassify	63.60	58.51	66.17	60.34	55.24	58.71
BBN HC	76.57	81.32	69.94	77.59	83.62	68.09
BBN TABU	76.57	81.32	69.94	77.59	83.62	68.09
BBN TAN-CL AIC	83.89	79.41	83.96	87.07	85.60	87.35
BBN TAN-CL BIC	77.82	76.08	77.31	83.62	82.22	81.24
BBN TAN-CL loglik	84.94	78.54	88.25	86.21	83.16	86.16
BBN FSSJ	80.75	81.10	76.05	83.62	88.43	76.17
BBN BSEJ	76.15	71.15	75.63	80.17	74.63	79.37
BBN HC-TAN	76.15	69.05	77.03	82.76	78.35	80.86
BBN k-DB	80.96	74.90	79.63	86.21	82.69	85.46
BBN HC-SP-TAN	77.62	72.88	78.43	83.62	79.97	81.26

. The comparative parameters are the ACC classification accuracy and the average values of PPV precision and TPR sensitivity.

The following conclusions were drawn from the comparison of the results:

• The highest scores for a training set were obtained for the PNN and SVM methods, but the results for the test set were not good enough;
• The results for the NBC method were very irregular and depended on the used package; the best of them was comparable with the BBN method;
• The BBN methods allow for obtaining the best results for the test set, and their results for the training set are good.

7. Conclusions

The presented research illustrates the effectiveness of selected ML methods for damage risk assessment in buildings subjected to external influences from the industrial environment generated by underground mining operations. The methods have been selected on the basis of the specifics of each method so that it is possible to assess the risk of damage in a probabilistic form. A gradation of methods was made from the least to the most complex. As a result of multiple analyses, results have been obtained, allowing us to state the effectiveness of each of the applied methods.

The best results of the research were obtained with the use of the Bayesian belief network machine learning method with the TAN-CL AIC structure learning method.

The selection of the BBN-based methodology as the most effective approach broadens the scope of applying the obtained model for practical purposes. In addition to generating responses on the level of risk of failure, it allows for the revision of the resulting cause-and-effect connections and the use of such a model in diagnostic cases, which very often occur in practice and are aimed at determining the causes of the failure state. In addition, such a system is very flexible in terms of computation because it is fed with new data and easily updates its parameters. Therefore, it makes it a potentially very effective tool for the real-time damage risk assessment and allows it to be used in a broader sense as a component of the SHM (structural health monitoring) [77] system operating in accordance with the IoT (Internet of Things) ideology [78].

In the course of further work, studies on the impact of damaged masonry walls on the building’s energy demand are taken into account. The damage categories used in the research have a different impact on the safety of building use, their energy demand and additional CO₂ emissions, which requires their further verification.