Performance Evaluation of Artificial Neural Networks (ANN) Predicting Heat Transfer through Masonry Walls Exposed to Fire

Featured Application

The use of Artificial Neural Networks for the prediction of heat transfer through a variety of masonry wall build-ups exposed to elevated temperatures οn one side.

Abstract

The multiple benefits Artificial Neural Networks (ANNs) bring in terms of time expediency and reduction in required resources establish them as an extremely useful tool for engineering researchers and field practitioners. However, the blind acceptance of their predicted results needs to be avoided, and a thorough review and assessment of the output are necessary prior to adopting them in further research or field operations. This study explores the use of ANNs on a heat transfer application. It features masonry wall assemblies exposed to elevated temperatures on one side, as generated by the standard fire curve proposed by Eurocode EN1991-1-2. A juxtaposition with previously published ANN development processes and protocols is attempted, while the end results of the developed algorithms are evaluated in terms of accuracy and reliability. The significance of the careful consideration of the density and quality of input data offered to the model, in conjunction with an appropriate algorithm architecture, is highlighted. The risk of misleading metric results is also brought to attention, while useful steps for mitigating such risks are discussed. Finally, proposals for the further integration of ANNs in heat transfer research and applications are made.

1. Introduction

1.1. Purpose and Innovation of This Work

Machine Learning and Artificial Neural Networks, in particular, are increasingly becoming the scientific method of choice for several engineering researchers and practitioners alike. Although the notion was introduced in 1956 [1], its practical adoption was delayed until the early 1990s [2] mainly due to the inadequate computational power available.

Arguably, the benefits Artificial Neural Networks (ANNs) can bring to a scientific project make them an attractive method for analysing data. Enabling researchers to make predictions regarding the phenomenon they study without the need for computationally heavy numerical models or expensive testing facilities, in conjunction with Finite Element or BIM model analysis [3], significantly reduces the cost of their research. In the same direction, the reduction in resources used for experiments and the ability to draw information from previous experimental work could put the use of ANNs at the forefront of efforts for sustainability within the scientific community. In terms of reliability and performance, it has been shown that, usually, ANNs outperform or are of equivalent accuracy to traditional linear and nonlinear statistical analysis methods [4]. Despite these apparent advantages, the adoption of ANNs in certain research fields, such as heat transfer and fire engineering in the context of civil engineering, is still slow [5].

This study’s contribution is twofold; it aims to fill part of the ANN utilisation gap in heat transfer research, on one hand, while stimulating a constructive dialogue regarding the evaluation and significance of input selection for ANNs on the other. The first aim is achieved through the development of an ANN architecture able to make predictions on the thermal performance of masonry wall assemblies exposed to elevated temperatures. It specifically proposes a neural network algorithm capable of predicting the temperature development on the non-exposed face of the wall assemblies over time, when the standard fire curve (as described in Eurocode EN1991-1-2) is applied to the other face of the wall. Although there are a number of studies exploring the use of Machine Learning and Neural Networks in applications such as structural health monitoring [6] and the residual capacity of structural members [7], the adoption of these methods is still not as widespread within the field of fire engineering as in other scientific domains [5]. Equally, there are studies exploring the optimisation of the internal environment for user comfort through the use of Artificial Intelligence [8]; the main focus, though, largely deviates from heat transfer through structural members. The first part of this study’s unique contribution is that it combines aspects of fire engineering (exposure to fire and the change of phase of the components of the structural member) with the transient movement of heat through a masonry wall. This work paves the way for a range of practical applications of heat transfer through the composite, non-uniform, and perforated elements of varying material thermal properties with prospects in the field of civil engineering and materials science.

The latter is approached with a systematic evaluation of the aforementioned algorithm’s performance in terms of accuracy and reliability. Different metrics are used to measure the credibility of the ANN in this specific application and an intuitive comparison to the “ground truth” data is also employed to evaluate the final predictions. To ensure a robust model development and evaluation process, a step by step approach is adopted, closely following previously published ANN development protocols as a guideline [9]. The developed evaluation methodology constitutes an initial step towards the establishment of a standardised process of quantifying the input data impact on the ANN’s performance. A parallel investigation of a wide range of research facets (other ML techniques, optimisation algorithms, the integration of other scientific fields beyond fire engineering, and heat transfer, to name a few) could provide additional value to the current proposal. Nonetheless, this initial attempt already contributes towards building confidence around the accuracy and reliability of ANN results, and it sets an initial foundation for developing a filter and evaluation process during the selection of input datasets through a quantitative and visual assessment of ANN predictive performance.

Several scientific efforts have focused on optimising and assessing the impact of alternative algorithm architectures and the choice of hyperparameters on the performance of ANNs [10]; on the other hand, this study explores the impact of varying degrees of input data quantity and quality. The initially developed ANN architecture is maintained identically throughout the study, while portions of the input data are gradually withheld, with the intention to observe the degradation of the ANN’s predictive capabilities. This results in four different regressor models with identical structures but significantly different ultimate performance. The predictions of each model are assessed and the risk of receiving misleading results is commented upon. Eventually, conclusions and recommendations on how to mitigate such risks arising from specific ANN metrics are made.

Arguably, the field of Machine Learning is constantly evolving and expanding to incorporate new bodies of theoretical study and practical applications. An extensive and analytical review of such works would undeniably be fruitful and beneficial. Nevertheless, the nature of such a thorough investigation does not align directly with the objectives of the current study. Cutting edge research currently attempts to optimise the architecture and performance of Machine Learning algorithms by exploring solutions ranging from the use of genetic algorithms [11] and dispersing the computational and data processing load to peripheral computers (the Edge) [12], to developing state of the art spatial accelerators to expedite big data processing [13]. Despite the utmost significance of these advancements, this study aims to establish an evaluation regime for the more conventional form of ANNs, which are still in the early stages of their implementation within the field of fire engineering and built-environment heat transfer applications.

1.2. Advantages and Disadvantages of the Proposed Evaluation Method

Artificial Neural Networks are progressively used more extensively both in the context of scientific research and industrial applications. Despite the multifaceted benefits their use can bring to the research and industrial processes, careful consideration needs to be given to achieve a balanced assessment of their advantages and drawbacks. The following list outlines some of the main advantages of using ANNs within the context of heat transfer and fire engineering and highlights the benefits of the methodology proposed herein in terms of enhancing these favourable attributes:

• The use of ANNs can lead to a reduction in the cost and resources associated with further fire and heat transfer testing. This also contributes towards achieving the sustainability targets of research institutions and academic organisations by removing the need for the unnecessary use of fuel, test samples, and the construction of testing facilities. The proposed methodology provides an opportunity for a more efficient and accurate assessment of the ANN’s results.
• Artificial Neural Networks can also help reduce the required time for developing and analysing computationally heavy 3D Finite Element heat transfer models. This further reduces the cost associated with acquiring powerful computers to support the aforementioned models. Despite this powerful feature, the use of ANNs needs to be regulated and evaluated; the methodology developed herein sets the foundation for such an evaluation framework.
• ANNs can increase and simplify the reproducibility of heat transfer and fire performance experiments. ANNs introduce efficiency in the exploratory amendment of experiment parameters. Previously recorded figures can feed in and help construct the ANN model, providing the ability to then tweak and replicate the parameters to explore variations of the original experimental arrangements. As this process involves the amendment and adjustment of the input data of ANNs, the workflow introduced in this study can highlight the potential pitfalls arising from this retrospective processing.
• The methodology for input data review proposed herein aims to contribute towards the prevention of misleading or not adequately precise results generated by the ANN, for integration into further research or field applications.
• This body of work also raises awareness regarding the need for a standardised methodology for assessing ANN performance and ANN model development.

On the other hand, some of the main disadvantages and limitations that need to be addressed before ANNs and the proposed assessment methodology are fully integrated into the scientific and industrial workflow include:

• The development of ANNs usually requires a large amount of input data, which are not always easy or affordable to obtain. The proposed methodology essentially introduces additional filters and methods for stricter input data quality assessment, which can make the above process even more involved.
• The interpretation of the ANN output is not always straightforward, and its precision can be difficult to judge unless a solid understanding of the expected results has been developed beforehand. The following sections of this study touch on this specific matter and make recommendations.
• To extend the scope of the scientific and industrial application of the proposed model within the context of the heat transfer and fire performance of wall assemblies, further research is needed to incorporate a wider range of material properties, fire loadings, and geometrical wall configurations.
• There are a plethora of proposed model architectures and ML approaches that could potentially enhance the performance of the proposed model. Although the detailed review and integration of such methodologies fall outside the scope of the present study, it would be beneficial for those to be reviewed and compared against the proposed model structure and topology.

1.3. Scope Limitations and Extended Application

The evaluation process presented in the following paragraphs focuses on a specific application (heat transfer through masonry wall elements exposed to fire on one side) and, as such, it is optimised for the parameters describing this particular case. The parameters used as part of this study could be extended to capture additional features of the experimental and/or modelling data. This would provide a wider scope for the application of the evaluation methodology within the context of heat transfer and fire engineering. The additional parameters could include, but not be limited to, the moisture content of the wall assembly materials, the different types and positions of insulation, a range of different fire loads, ventilation parameters, the different compositions of masonry units with the associated variations in their thermal properties, and the different qualities of bedding and render mortar. The inclusion of a wider array of material properties, geometries, and thermal loading could greatly increase the scope of the modelling process and evaluation method.

Another aspect that could enhance the flexibility of the current proposal would be the integration of other Machine Learning (ML) techniques. Although an extensive comparative study is beyond the scope of the present work, a replication of the proposed process on the output generated by other ML algorithms and models would help build a more holistic understanding of the method’s transferability.

Although there is wide scope for the further expansion of the proposed methodology, a conscious effort has been made to primarily focus on its underpinning principles. This provides an overview of the cornerstones of the evaluation concept and enables a straightforward transfer of its universally applicable values. The aim of the final conclusions and recommendations is to furnish fellow researchers with a guideline on how to assess their ANN output regardless of their specific scientific field and practical application.

1.4. Intuition of Artificial Neural Networks

Artificial Neural Networks (ANN) belong to the sphere of supervised Machine Learning (ML), which in turn falls under the wider concept of Artificial Intelligence (AI) [14]. The cornerstones of ANNs are their architecture and training data. In terms of architecture, each ANN comprises multiple layers of interconnected nodes. These include the input layer (input nodes), responsible for receiving and offering the input data to the network; the output layer (output node), which represents the final prediction of the network; and the hidden layer(s) in between, which capture and factor the various features present in the available dataset. In the case of Deep Learning, the ANN category that the present study falls under, the networks involve more than one hidden layer. These are fully connected; all nodes of one layer are connected to all nodes of the previous and following layers.

The structure of ANNs resembles the structure of the human brain. Each node, mentioned above, represents a neuron, and the connections between those perform similar functions to brain synapses, as seen in Figure 1. Each neuron represents a feature of the training data and can be outlined by Equation (1), where f(x) is an appropriately selected activation function [15,16]:

$$\mathrm{P}=\mathrm{f}({\sum }_{i=1}^n{w}_i{x}_i+\mathrm{b})$$

(1)

where P is the output figure of the neuron, w_i is the weights applied to each feature of the data that are fed to the neuron, x_i is the features of the input data themselves, and b is the applied bias.

Each neuron is initially assigned a random weight, quantifying the significance of the represented feature to the value of the dependent variable. These weights are constantly updated and ultimately optimised through an iterative training process (epoch). In each training loop, the contribution of each neuron to the dependent variable and the overall accuracy of the ANN algorithm prediction against a known value is evaluated. The process of comparing the network predictions against known values (the ground truth) is the founding principle of supervised learning [17]. To enable this internal assessment and feedback process, the use of a loss function at the end of each epoch is necessary. This feedforward, backpropagation training process is illustrated schematically in Figure 2.

Similar to the architecture of the network, the choice of an appropriate loss function, along with parameters including, but not limited to, the activation function, number of epochs, learning rate, and batch size, is dependent on the type, quantity, and quality of the input data and type of expected predictions [18]. These are called hyperparameters, and in conjunction with the ANN structure (number of hidden layers and number of neurons on each hidden layer) turn an ANN algorithm into a trainable and functioning predictive model. There is no prescriptive method for standardising the process of ANN development at the moment, in terms of hyperparameters and architecture selection [19]. However, previous scientific studies summarise useful conclusions that provide some guidance in this respect.

Apart from developing an effective architecture and choosing appropriate hyperparameters, an aspect that requires attention is network overfitting [1]. Although an algorithm can perform extremely well in making predictions on the given training and test set, there is the risk of excessive focus on the provided observations, leading to the reduced general applicability of the model. Overfitting limits the scope of use of an artificial neural network and can potentially lead to misleading prediction results. The impact of input data quality and quantity on the overfitting of the ANN is discussed in subsequent sections.

Fundamental details regarding the architecture of the ANN built and utilised for this study are presented in the following paragraphs, allowing for a more holistic understanding of the factors impacting the performance of the model. Similarly, the structure of the input and ground truth data is thoroughly explained in the relevant chapter of this paper, provoking thought on the relationship between input and output precision.

1.5. Input Data Reference

The development, implementation, and evaluation of the ANN algorithm constitute the focal point of this work. The basis of the heat transfer input data, used to train the ANN algorithm, became available after adjusting and interrogating finite element analysis models developed as part of previous scientific research. Specifically, Kanellopoulos and Koutsomarkos [20] set the foundation with their research on heat transfer through clay brick walls, focusing on the contribution of radiation through the voids of the masonry units. This was further developed by Kontoleon et al. [20,21] and their work on heat transfer through insulated and non-insulated masonry wall assemblies. The various amendments and pre-processing routines imposed on this foundation dataset are described in the following paragraphs.

1.6. Software and Hardware Utilised for This Research

The analysis of the physical phenomenon of heat transfer was undertaken using COMSOL Multiphysics^® simulation software (v 5.3a). The development, training, and review of the ANN algorithm were carried out using the programming language Python and the application programming interface Keras along with its associated libraries. Other libraries used include Pandas, NumPy, Statistics, Scikit-learn [22], Matplotlib, and Seaborn (the last two for visualisation purposes). Finally, the ANN predictions were exported to MS Excel for ease of diagram formatting and presentation.

The finite element analysis and the development and training of the ANN were performed using a 64-bit operating system, running on an Intel Core i7-920 processor at 2.67 MHz, and with 24GB of RAM installed.

2. Modelling and Methods

2.1. Masonry Wall Assembly Finite Element (FE) Models

2.1.1. Geometrical Features of Models’ Components

The training of the ANN algorithm was based on data extracted from 30 different wall assembly FE models. Despite having a core model onto which the structure of the wall samples was founded, a range of variables was incorporated to enable a more accurate representation of the phenomenon of heat transfer through the wall samples. This pluralism also enhanced the understanding of the impact of various parameters on the process, through the use of ANNs in the following stages of the study.

The foundation model involved the use of a single skin of perforated clay bricks stacked with cement bedding mortar. In the simplest modelling case, either face of this brick core was covered with a single layer of cement render before it was exposed to fire on one side (details regarding the boundary conditions and fire load applied to the model are included in the following paragraphs). Specifically, the masonry core consisted of 250 mm wd × 140 mm dp × 330 mm lg clay bricks incorporating an array of 18 full-length holes (12 holes of 26 mm × 34.7 mm and 6 more elongated holes of 26 mm × 56 mm), as indicated by the following sections in Figure 3. The bedding mortar and cement render were consistently 10 mm thick.

Two variations of this basic matrix model were then developed and analysed; namely, a brick wall insulated with EPS internally (exposed to fire) and a brick wall insulated with EPS externally (non-exposed face of the wall). For each case, two EPS layer thicknesses were analysed: 50 mm and 100 mm.

2.1.2. Model Material Properties and Combinations

The range of material properties incorporated into the FE models enriched the available analysis cases and consequently provided a wide variety of output data. The properties and their values considered as part of the FE analysis included:

• Clay brick density (ρ): 2000 kg/m³ and 1000 kg/m³.
• Clay brick thermal conductivity coefficient (λ): 0.8 W/(m·K) and 0.4 W/(m·K).
• Clay brick thermal emissivity coefficient (ε): 0.1, 0.5, 0.9.
• Insulation thickness (d): 0 mm, 50 mm, 100 mm.

The above range of variables and material property values allowed for the formation of the following combinations. Each combination appearing in

Table 1. Material property combinations defining the various FE models analysed.

Insulation Position and Thickness	Brick Density	Thermal Conductivity Coefficient	Thermal Emissivity Coefficient	Sample Reference 1
No insulation	1000	0.4	0.1	Smpl1-1
	1000	0.4	0.5	Smpl1-2
	1000	0.4	0.9	Smpl1-3
No insulation	2000	0.8	0.1	Smpl2-1
	2000	0.8	0.5	Smpl2-2
	2000	0.8	0.9	Smpl2-3
Non-exposed EPS (50 mm)	1000	0.4	0.1	Smpl3-1
	1000	0.4	0.5	Smpl3-2
	1000	0.4	0.9	Smpl3-3
Non-exposed EPS (50 mm)	2000	0.8	0.1	Smpl4-1
	2000	0.8	0.5	Smpl4-2
	2000	0.8	0.9	Smpl4-3
EPS exposed to fire (50 mm)	1000	0.4	0.1	Smpl5-1
	1000	0.4	0.5	Smpl5-2
	1000	0.4	0.9	Smpl5-3
EPS exposed to fire (50 mm)	2000	0.8	0.1	Smpl6-1
	2000	0.8	0.5	Smpl6-2
	2000	0.8	0.9	Smpl6-3
Non-exposed EPS (100 mm)	1000	0.4	0.1	Smpl7-1
	1000	0.4	0.5	Smpl7-2
	1000	0.4	0.9	Smpl7-3
Non-exposed EPS (100 mm)	2000	0.8	0.1	Smpl8-1
	2000	0.8	0.5	Smpl8-2
	2000	0.8	0.9	Smpl8-3
EPS exposed to fire (100 mm)	1000	0.4	0.1	Smpl9-1
	1000	0.4	0.5	Smpl9-2
	1000	0.4	0.9	Smpl9-3
EPS exposed to fire (100 mm)	2000	0.8	0.1	Smpl10-1
	2000	0.8	0.5	Smpl10-2
	2000	0.8	0.9	Smpl10-3

¹ Only used for ease of reference in the following sections of this study and for cross-referencing with model analysis files by the research team.

represents a separate FE model whose analysis output subsequently offered portions of the overall input data for training, testing, and evaluating the ANN model.

The development of a complete finite element analysis model required the definition of some additional parameters (which were not explicitly used for the development of the ANN). The specific heat capacity (C_p) of all components needed to be specified, along with the density (ρ) and thermal conductivity coefficient (λ) of the cement mortar and insulation panels.

Table 2. Other material properties used for the development of the FE models.

Material	Density kg/m3	Thermal Conductivity Coefficient W/(m∙K)	Specific Heat Capacity J/kg∙K
Clay bricks	As above	As above	1000
Insulation (EPS)	30	0.035	1500
Cement mortar	2000	1.400	1000

summarises this additional information.

It is worth highlighting that although most of the thermophysical and mechanical properties of the masonry structure fluctuate depending on the temperature of the components [23], for the needs of this study they have all been assumed to remain constant throughout the development of the phenomenon. It has also been shown that clay brick spalling affects the thermal performance of the masonry wall when exposed to fire [24,25]; this parameter has not been considered in the finite element models. The dramatic impact on the insulation’s performance, due to phase change when exposed to high temperatures, could not be ignored. The modelling convention used to reflect this characteristic behaviour is explained in the following paragraphs.

2.2. Heat Transfer Analysis, Fire Load, Assumptions, and Conventions

To effectively assess the performance of the ANN and evaluate the accuracy of its predictions, a basic description and understanding of the physical phenomenon under consideration are deemed necessary. The founding principles and equations of heat transfer are presented herein, in conjunction with the various assumptions, simplifications, and conventions used when constructing the relevant finite element analysis model.

2.2.1. Heat Transfer Fundamentals

Although the aim of this study is not to review and present the fundamental principles of heat transfer, it is worth briefly presenting the basic mechanisms operating on the finite element analysis model. The analysis commences with the wall samples in thermal balance, with an ambient room temperature applied on either side. At time t = 0 sec, an increasing fire load (as described in the following paragraph) is applied on one side (hereon referred to as “exposed”), initiating the combined heat transfer mechanism of convection and radiation. The fundamental Equations (2) and (3) govern the convective and radiation heat transfers, respectively [26], from the fire front towards the exposed wall surface:

$$q_{conv}^{″}=h \left(T_S-T_{\infty }\right)$$

(2)

$$q_{rad}^{″}=\epsilon ·\sigma \left(T_s^4-T_{sur}^4\right)$$

(3)

where $q_{conv}^{″}$ and $q_{rad}^{″}$ are the resulting heat flux due to convection and radiation, respectively, h is the convection heat transfer coefficient (W/(m²·K)), T_s is the surface temperature, T_∞ is the surrounding fluid temperature (for convection), T_sur is the surrounding environment temperature (for radiation), ε is the emissivity coefficient, and σ is the Stefan–Boltzmann constant (σ = 5.67 × 10⁻⁸ W/(m²·K⁴)).

The heat transfer mechanism of conduction is also activated as heat progressively travels through the solid parts of the wall assembly. Equation (4) is the governing relationship for heat conduction, and it was used to replicate the phenomenon on the finite element analysis model. The clay brick cavities play an integral part in the transfer of heat through the sample [27], and thus attention was paid to modelling them accurately. Heat transfer through radiation between the cavity walls and convection through air movement within the cavities has been considered. The previously described convection and radiation formulas apply:

$$q_{cond,x}^{″}=-\lambda \frac{\partial T}{\partial x}$$

(4)

where $q_{cond,x}^{″}$ is the resulting heat flux due to conduction, λ is the thermal conductivity, and $\frac{\partial T}{\partial x}$ is the temperature gradient in the direction of the heat’s transient movement.

The final stage of the transient movement of heat through the wall sample is its release to the environment through the non-exposed face of the section. The applicable heat transfer mechanisms are convection, through the surrounding air, and radiation.

2.2.2. Boundary Conditions and Fire Load

A definition of the applicable boundary conditions is necessary before the solution of the differential equations can be attempted. The analysis initiates assuming an ambient room temperature of 20 °C on both faces of the wall assembly. This condition is adhered to throughout the analysis for the non-exposed wall face.

At time t = 0 sec, the “exposed” wall face becomes subject to the increasing fire load represented by the standard fire curve ISO 834 as present in Eurocode EN1991-1-2 [28]. Although scientific research is being carried out to identify and compile new, more accurate fire curves and loading regimes [29], it was considered that the well-established methodology proposed by the Eurocodes currently fulfils the needs of this study. Equation (5) mathematically describes the relationship of temperature increase due to the applied fire load over time, while Figure 4 is the corresponding visual representation:

Qg = 20 + 345 log10 (8 t + 1)

(5)

where t is the time in seconds and Q_g is the developed temperature in °C.

The boundaries between the various layers of the wall assembly follow the fundamental heat transfer equations, as described in the previous paragraph.

2.2.3. Modeling Heat Transfer Assumptions and Conventions

An element that required special attention was the behaviour of EPS when exposed to significantly high temperatures. The material is a combustible thermoplastic that remains stable when exposed to temperatures up to 100 °C. Its thermal properties start to rapidly degrade shortly after that, while temperatures upwards of 160 °C lead to the complete loss of the inflated bead structure and the liquefaction of the insulation boards. Approaching temperatures of 278 °C leads to the gasification of the material [30]. Provided that the wall samples were subject to temperatures significantly higher than 100 °C, the above behaviour had to be stimulated.

Conventionally, and accepting that no material can be removed from the finite element analysis model once the analysis has commenced, a temperature-dependent variable was utilised to reproduce the reduced performance and ultimate collapse of EPS. The thermal conductivity coefficient (λ_EPS) of the insulating material was artificially increased when temperatures beyond 150 °C were encountered. That allowed for the unhindered transfer of heat through the melting EPS boards, resembling the gradual removal of the physical barrier. The coefficient was linearly increased from a value of λ_EPS = 0.035 W/(m∙K) at 150 °C to λ_EPS = 20 W/(m∙K) at 200 °C (practically no heat transfer resistance). Similarly, its density (ρ_EPS) and special heat capacity (Cp_EPS) were reduced to negligible figures. Figure 5 demonstrates the steep material property changes for the melting EPS layer.

It is worth highlighting that the failure of the EPS material would naturally result in the detachment and collapse of the attached render. To ensure the removal of the render’s beneficial insulating function, a similar approach was adopted, wherein all associated thermophysical properties were altered to reproduce its collapse. To ensure the change was introduced in a timely manner, an exploratory analysis was performed to identify the time for the interface between render and EPS to reach the critical temperature of 150 °C. This threshold temperature was achieved at t = 240 sec. The thermal conductivity coefficient (λ_render) was increased from λ_render = 1.40 W/(m∙K) to λ_render = 2000 W/(m∙K), while its density (ρ_render) and specific heat capacity (Cp_render) were reduced to negligible values.

Although the same process would normally apply to both faces of the wall (exposed to fire and non-exposed), only the properties of the exposed insulation and render were altered within the scope of the present study. Similarly, no allowance has been made for the ignition of EPS volatiles.

2.3. ANN Complete Input Dataset (CID)

Since part of the algorithm’s structure (the size of the input layer) depends on the structure of the input dataset, it was considered important to conclude the data format prior to commencing the development of the ANN itself. A brief outline of the parameters used has been given in the description of the FE models; these were further refined and structured in an appropriate CSV file format ready for reading by the algorithm.

The file incorporated columns representing the independent variables defining the FE models described in previous paragraphs. A “timestamp” column was also added to allow for the observation and correlation of the temperature magnitude on the non-exposed face of the wall over time as the phenomenon developed. The last column of the dataset included the output of the FE model analysis, the temperature observed on the non-exposed face of the wall at a 30 sec time step when the standard Eurocode fire curve was applied to the other face.

Table 3. Representative example of the dataset structure and contents.

Index	Sample Ref	Brick Density	Thermal Conductivity Coef.	Thermal Emissivity Coef.	Insulation Thickness	Insulation Type	Insulation Position	Time	Temperature of Non-Exposed Face
…	…	…	…	…	…	…	…	…	…
11,449	Smpl6-1	2000	0.8	0.1	50	EPS	Int	18,990	41.77246
11,450	Smpl6-1	2000	0.8	0.1	50	EPS	Int	19,020	41.85622
11,451	Smpl6-1	2000	0.8	0.1	50	EPS	Int	19,050	41.94014
11,452	Smpl6-1	2000	0.8	0.1	50	EPS	Int	19,080	42.02421
11,453	Smpl6-1	2000	0.8	0.1	50	EPS	Int	19,110	42.10843
…	…	…	…	…	…	…	…	…	…

provides a typical section of the dataset file offered to the ANN algorithm; the example reflects the values used for an internally insulated wall panel.

It is apparent that not all columns were necessary for the training, testing, and evaluation of the algorithm, thus the input data was further modified and stripped into a more appropriate format as part of the “preprocessing” phase (see following paragraphs). Nevertheless, these variables and references are useful for an intuitive understanding of the input data content and structure. Variables referring to material properties were discussed in detail in previous paragraphs. Other columns include:

• Index—variable purely counting the number of unique observations included in the data. Each temperature measurement generated by the FE model analysis (time step of 30 s) is used as a separate observation. The complete dataset includes a total of 21,630 observations. The index was excluded from any training or testing of the algorithm.
• Sample reference—this variable enabled the research team to easily cross-reference between the dataset tables and the analysis files. Similarly, it was excluded from any training or testing of the neural network.
• Insulation type—the data structure and ANN algorithm were developed with the intention of incorporating and analysing various insulation materials. Although the present study only considers EPS, it was considered useful to build some flexibility in the algorithm, enabling the further expansion of the scope of work in the future. It takes the values “EPS” and “NoIns”, representing the insulated and non-insulated wall samples, respectively. This categorical variable was encoded into a numerical one as part of the pre-processing phase.
• Insulation position—this variable represents the position of the insulation. It takes three values; “Int”, “Ext”, and “AbsIns”, representing insulation exposed to fire (internal insulation), insulation not exposed to fire (external insulation), and non-insulated wall samples, respectively. As a categorical variable, this was also encoded as part of the pre-processing phase.
• Time—to enable the close observation of the temperature development on the non-exposed face of the wall over time, it was considered necessary to include a “timestamp” variable. This was obtained directly from the FE model analysis output, where time and temperature are given as two separate columns. The temperature is measured every 30 s for a duration of 6 h and a total of 720 observations for each of the 30 models.
• Temperature of the non-exposed face—this is the output of the finite element analysis models in °C as generated by COMSOL Multiphysics^® simulation software. It reflects the temperature developed gradually within a reference area on the non-exposed face of the wall panels. This constitutes the dependent variable of the dataset and ultimately is the figure that the ANN algorithm will be trying to predict in the following steps.

It is apparent that the above set of parameters and values are relevant only to the specific masonry wall heat transfer application presented in this study. However, the process of compiling a group of independent variables and organising them into a dataset structure appropriate for analysis in Python is universal. Depending on the number of the recorded independent variables, the architecture of the network might need to be adapted (more or fewer input neurons), different hyperparameters might generate better results, or slightly different preprocessing methods might be applicable (scaling might/might not be necessary, encoding of categoric variables might be needed or not, etc). The list of used variables, and their range of values given in the preceding Table 1 and Table 3, should provide a guide for understanding the form of the dataset and possibly substituting it with other data available to interested research parties. Similarly, the list of hyperparameters included in the following sections of the study, along with the values used for this research, should provide adequate detail for understanding and replicating the structure of the ANN itself, if desired.

2.4. Test Cases Examined

Part of this study’s unique contribution is to examine the impact of varying degrees of input data quantity and quality on the performance of the ANN model. An attempt was made to isolate and assess the influence of data by keeping the same algorithm architecture and gradually altering the amount of information provided for training. This provided a level ground for comparing the algorithms, without introducing inconsistencies due to hyperparameter and architecture variations.

Table 4. List of wall assembly analysis output used for training each algorithm.

Sample Reference	Properties of Wall Sample	ANN 1	ANN 2	ANN 3	ANN 4
Smpl1-1	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.1	✓	✓	✓	✓
Smpl1-2	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.5	✓	✓
Smpl1-3	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.9	✓	✓	✓	✓
Smpl2-1	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.1	✓	✓	✓	✓
Smpl2-2	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.5	✓	✓
Smpl2-3	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.9	✓	✓	✓	✓
Smpl3-1	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.1, d = 50 mm, External	✓		✓
Smpl3-2	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.5, d = 50 mm, External	✓
Smpl3-3	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.9, d = 50 mm, External	✓		✓
Smpl4-1	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.1, d = 50 mm, External	✓		✓
Smpl4-2	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.5, d = 50 mm, External	✓
Smpl4-3	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.9, d = 50 mm, External	✓		✓
Smpl5-1	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.1, d = 50 mm, Internal	✓		✓
Smpl5-2	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.5, d = 50 mm, Internal	✓
Smpl5-3	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.9, d = 50 mm, Internal	✓		✓
Smpl6-1	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.1, d = 50 mm, Internal	✓		✓
Smpl6-2	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.5, d = 50 mm, Internal	✓
Smpl6-3	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.9, d = 50 mm, Internal	✓		✓
Smpl7-1	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.1, d = 100 mm, External	✓	✓	✓	✓
Smpl7-2	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.5, d = 100 mm, External	✓	✓
Smpl7-3	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.9, d = 100 mm, External	✓	✓	✓	✓
Smpl8-1	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.1, d = 100 mm, External	✓	✓	✓	✓
Smpl8-2	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.5, d = 100 mm, External	✓	✓
Smpl8-3	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.9, d = 100 mm, External	✓	✓	✓	✓
Smpl9-1	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.1, d = 100 mm, Internal	✓	✓	✓	✓
Smpl9-2	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.5, d = 100 mm, Internal	✓	✓
Smpl9-3	ρ = 1000 kg/m3, λ: 0.4 W/(m∙K), ε = 0.9, d = 100 mm, Internal	✓	✓	✓	✓
Smpl10-1	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.1, d = 100 mm, Internal	✓	✓	✓	✓
Smpl10-2	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.5, d = 100 mm, Internal	✓	✓
Smpl10-3	ρ = 2000 kg/m3, λ: 0.8 W/(m∙K), ε = 0.9, d = 100 mm, Internal	✓	✓	✓	✓

summarises the input data used for training each of the 4 ANN models. The original algorithm (ANN 1) was developed using the full dataset, comprising the entirety of the data obtained through the FE analysis, as described in detail in previous paragraphs. Each subsequent algorithm was trained with a subset of the original input information. Specifically, the cases examined include:

• ANN 1: As mentioned above, this uses the complete dataset for training and testing purposes.
• ANN 2: The second algorithm was developed using only the extreme values of insulation thickness. As such, the wall assemblies considered included the non-insulated ones and those insulated with 100 mm of EPS internally and externally.
• ANN 3: Only the extreme values of the emissivity coefficient were used for the development of the third algorithm. Wall assemblies with ε = 0.5 were disregarded and only those with ε = 0.1 and ε = 0.9 were included in the dataset.
• ANN 4: This was the most input data-deprived algorithm—a combination of the previous two cases. Only the extreme cases of insulation and thermal emissivity coefficient were offered to the algorithm at the training stage, considerably reducing the density of the offered input data.

Each algorithm was eventually compared to the values included in the full set of information, with the aim of identifying the level of inaccuracy introduced by withholding part of the input data. The comparison was carefully made against the wall assemblies incorporating the variable values that the algorithms were deprived of. Since the regressor models were generally trained using extreme values of insulation (with the exception of ANN 1, which utilised the full dataset), the comparison was made against wall assemblies featuring mid-range values (i.e., 50 mm of insulation or ε = 0.5). Although it was anticipated that ANN 1 would have an extremely good predictive score (since it was already trained with full data), it was included in the resulting graphs for comparative reasons.

In addition to assessing each algorithm’s performance, it was important to ensure that overfitting was avoided. Once the most accurate regressor was identified, it was considered necessary to evaluate its applicability on wall assemblies that had not been offered to the algorithm at any stage of its development and were, as such, completely unknown to the model. The variable values and basic structure of these new models had to be within the range of the algorithm’s training space, as shown on

Table 5. Additional FE model parameters.

Sample Ref	Brick Density	Thermal Conductivity Coef.	Thermal Emissivity Coef.	Insulation Thickness	Insulation Type	Insulation Position
Test Sample 1	2000	0.8	0.9	25	EPS	Ext
Test Sample 2	2000	0.8	0.9	25	EPS	Int
Test Sample 3	2000	0.8	0.7	0	NoIns	AbsIns
Test Sample 4	2000	0.8	0.3	0	NoIns	AbsIns
Test Sample 5	1500	0.6	0.9	0	NoIns	AbsIns
Test Sample 6	1500	0.6	0.7	75	EPS	Int

; however, they featured completely new parameter value combinations. To enable the comparison of the ANN predictions with actual ground truth values, six new FE models were developed and analysed. Their output was finally compared to the regressor predictions, as shown in the results section of this study. The six new models included the following:

2.5. ANN Development Protocol

It is a common realisation of the scientific community (at least, the parts working with Machine Learning and Artificial Neural Networks, in particular) that no formal standards, guidance or commonly accepted scientific methods of developing an ANN are available at the moment [9]. Efforts to establish such methods have indeed been made, providing the first step towards standardisation and a defined set of criteria for choosing the algorithm input data, architecture, training parameters, and evaluation metrics. The following paragraphs are dedicated to a step-by-step description of the ANN development process, following relevant, recently proposed protocols [9,31].

2.5.1. Input Data Selection and Organisation

As mentioned previously, data selection and organisation are the first steps towards the development of a functional and effective ANN. The importance of careful input selection is underlined, with a particular focus on data significance and independence. Although there are various statistical ways of determining the significance of data, the use of previous experience and domain knowledge is deemed acceptable and valid [9]. In this case, the independent variables selected are all known to have a considerable contribution to the development of the final values of the dependent variable.

Data filtering and clustering can reduce the number of input parameters, enabling the development of a more efficient and computationally manageable algorithm [32]. Such techniques ensure that any interdependence present in input variables is identified and either removed or merged, reducing the dimensionality of the necessary analysis. The number of independent variables used in the present study is considered small. Although, at present, some dependency between variables is known to exist (i.e., brick density with its thermal conductivity coefficient), the data is structured in a way that allows for the introduction of further test cases able to remove such issues (i.e., varying combinations of brick density and thermal conductivity that are not proportionally related).

2.5.2. Input Data Preprocessing

Although data preprocessing does not appear explicitly as a separate step in the ANN development protocol, it was considered necessary to be mentioned here; it did form an integral part of the work method followed for this study. From the input data presentation, it became apparent that categorical variables have also been included in the dataset (i.e., insulation type, insulation position, etc.). These variables were encoded to ensure that, ultimately, the input offered to the ANN was consistently numerical. To prevent the negative effect of a “dummy trap” [33], some of the resulting encoded numerical variables were removed.

Acknowledging that the range of the variable values was quite wide, taking values from 1 for the encoded variables to 21,600 sec for the timestamps, a scaling of the data was considered appropriate [34]. To prevent an implicit bias of the algorithm towards the higher values of the dataset, standardisation was applied throughout the dataset values. The default values of the “StandardScaler” function from the library sklearn.preprocessing were utilised [35]. Equation (6) describes the scaling method followed for both the input and output data. Once the predictions were made, the same scaling tool was used to reverse the scaling and allow for the inspection of the actual predicted temperature figures generated by the algorithm.

$$z=\frac{\left(x-u\right)}{s}$$

(6)

where z is the standard score of the sample being converted, x is the feature being converted, u is the mean, and s is the standard deviation.

2.5.3. Data Splitting

Splitting the dataset into training and testing sets is a fundamental part of the ANN algorithm development process. To avoid introducing figure selection bias, the use of automatic random selection was opted for: the “train_test_split” function from the “sklearn.model_selection” library, under the Scikit Learn API [35]. Random selection is currently the most-used unsupervised splitting technique [9]. Following the example of several other research studies, a ratio of 80–20% of the available observations was allocated to training and testing the algorithm, respectively [14,36].

2.5.4. Model Architecture and Structure Selection

The feedforward multilayer perceptron (MLP), the ANN setup that was utilised for this study, is one of the most popular artificial neural network architectures [31]. The number of input and output nodes was defined by the number of input and output variables, respectively. The input layer features 8 nodes, reflecting the number of input variables. This incorporates the dummy variables introduced due to encoding the categorical features, as mentioned previously. The output layer includes a single node representing the dependent variable, which the ANN has to make predictions against (non-exposed wall-face temperature through time). To achieve a Deep Learning model, a minimum of two hidden layers had to be constructed. The number of nodes on each layer was based on previous experience of ANNs achieving satisfactory prediction results. Figure 6 represents the structure of the ANN used as part of this study.

Although methods of optimising the architecture and hyperparameters of the model do exist, achieving the optimum model was beyond the scope of this study, which instead focused on the impact of the quality and quantity of input data. Given that an identical model architecture was utilised for all 4 different models, the final comparison was undertaken on a level plain. Since the activation function has an impact on the performance of the model [37], it is worth mentioning that the rectified linear unit activation function was used in both hidden layers of all models. For clarity, the other hyperparameters used for the development of the ANN are presented in

Table 6. Hyperparameters used for the development of the ANN.

Hyper Parameter	Value	Comments
Number of epochs	50	Following ad hoc experimentation with various values.
Learning rate	0.001	Default value of Adam optimiser.
Batch size	10	To allow more efficient processing of the large dataset.
Activation function	ReLU	Used the default values of the rectified linear unit activation function.
Optimiser	Adam	Default values of Adam [38] optimiser were used.
Loss function	MSE	Mean squared error loss function.

.

2.5.5. Model Calibration

Through training, the ANN converges to the optimum values of the initial random weights assigned to its neurons; this enables an accurate prediction of the dependent variable to eventually be made. One of the most computationally efficient algorithms for achieving such optimisation is backpropagation [39]. A loss function is used at the end of every epoch to calculate the difference between the ANN prediction and the ground truth. This study utilises the Mean Squared Error (MSE) loss function, since the physical phenomenon under consideration was not linear and the final predictions were not distinct categories (i.e., aiming for the regression of a range of values). With the level of inaccuracy now quantified, an optimisation function, in this instance Gradient Descent, calculates the local minimum (optimum) and feeds back to the network, updating the neuron weights accordingly to reduce the difference between the prediction and the truth (loss). Although other methodologies for optimising the performance of the model are available [40], they are beyond the scope of this study.

2.5.6. Model Validation

Once the ANN is constructed, trained, and functional, it is necessary to evaluate its performance before deploying it in field operations or using it for further scientific research. The evaluation process of the ANN constitutes the focal point of this study, and although it will be explained thoroughly in the results section, it is worth briefly mentioning the three core evaluation aspects, along with the various metrics to be considered. An outline of the implementation method of the above, within the context of this research effort, will also be given, with the intention of gradually introducing the structure of the upcoming results section.

There are three main steps in validating the functionality and reliability of an artificial neural network [9]. The “replicative validity” is the first thing to check to ensure that the ANN is functional and captures the underlying mechanisms of the physical phenomenon. Essentially, the algorithm needs to be able to replicate the known data observations that were offered as input (including the training and testing sets). This process yields “obvious” results, but it does also provide a sanity check that the algorithm has captured at least the minimum relationship between the independent and dependent variables. The use of fitness metrics or visual observation of comparative graphs between predictions and provided data can aid in this direction. In this study, both methods, metrics and visual inspection of the algorithm’s ability to replicate the data, have been employed.

The validation of the predictive capacity of the algorithm is the second stage in building confidence before its implementation in real applications. In this step, the ability of the algorithm to make accurate predictions, when provided with unknown (not included in the original training data set) input observations, is assessed by the use of specific efficiency indices. The impact of training some models with progressively fewer input data becomes apparent at this stage. An observation of the diminishing scores of the various metrics and also the deviation of the graphical representation of the predictions from the ground truth values elucidates the major contribution that appropriately rich and diverse input data can have on the development of an effective ANN.

Finally, the “structural validity” of the model needs to be confirmed. As part of this step, the neural network is checked against “a priori knowledge” of how the physical system under consideration works [9]. Apart from making correct predictions on specific values, the ANN needs to prove a holistic understanding of the underlying mechanisms that define the phenomenon that is being studied. In this study, instead of generating only individual predictions, the ANN is requested to predict the whole time series of the phenomenon. Thus, the structural validity of the ANN is evaluated through a comparison of the predicted physical behaviour against the known development of the heat transfer process through the various wall samples.

In the previous paragraphs, reference was made to the metrics used to assess the performance of the ANN in terms of accuracy and reliability.

Table 7. Metrics used for the evaluations of the performance of the ANN.

Metric	Reference	Formula 1	Perfect Score
Absolute maximum error	AME	AME = max(|$Q_i-{\widehat{Q}}_i$|)	0.0
Mean absolute error	MAE	MAE = $\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left|Q_i-{\widehat{Q}}_i\right|$	0.0
Relative absolute error	RAE	RAE = $\frac{{{\displaystyle \sum }}_{i=1}^n\left|Q_i-{\widehat{Q}}_i\right|}{{{\displaystyle \sum }}_{i=1}^n\left|Q_i-\overline{Q}\right|}$	0.0
Peak difference	PDIFF	PDIFF = max($Q_i$) − max(${\widehat{Q}}_i$)	0.0
Per cent error in peak	PEP	PEP = $\frac{\max \left(Q_i\right)-\max \left({\widehat{Q}}_i\right)}{\max \left({\widehat{Q}}_i\right)}\times$ 100	0.0
Root mean squared error	RMSE	RMSE = $\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n\left(Q_i-{\widehat{Q}}_i\right)}{n}}$	0.0
Coefficient of determination	R2	R2 = ${\left[\frac{{{\displaystyle \sum }}_{i=0}^n\left(Q_i-\widehat{Q}\right)\left(\widehat{Q}-\tilde{Q}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(Q_i-\overline{Q}\right)}^2{{\displaystyle \sum }}_{i=1}^n{\left({\widehat{Q}}_i-\tilde{Q}\right)}^2}}\right]}^2$	1.0

¹ Nomenclature of the above formulas: n = number of data points; $Q_i$ = observed value; ${\widehat{Q}}_i$ = ANN value prediction; $\overline{Q}$ = mean of the observed data points; $\tilde{Q}$ = mean of the values predicted by the ANN.

presents the main indices used as part of this work [41].

2.5.7. From Evaluation Methodology to Structured Results

The above outlines the main steps and methods for the performance and validity evaluation of the developed artificial neural network algorithms; it also informs the structure of the investigation and results of this study. Instead of exploring methods of algorithm optimisation, the research interest, in this case, focuses on the impact of quality and quantity of the provided input data. As such, a single algorithm was developed, adhering to the requirements of the “Architecture and Structure” section above. Then, the same algorithm was trained with varying numbers of input data (as presented in Section 2.4 Test Cases) and was thereon treated as 4 separate predictive models. Each model was subsequently evaluated, using the aforementioned indices and metrics, to reach a conclusion regarding the most effective regressor. As a final step, the dominant model was tested and evaluated again, against completely unknown data combinations. A brief outline of the work sequence followed so far and leading to the following results would include:

• An initial review of the existing bibliography proposing a protocol for the development of ANNs.
• The development of one ANN algorithm based on the peer-reviewed protocol.
• The training of the ANN algorithm with varying degrees of input data (ending up with 4 regressors of the same architecture but different levels of input data).
• A comparison of the performance of the 4 regressor models (same architecture/different input data) and an evaluation of the impact of the quality of offered input data on each model’s predictive capability.
• The identification of the best-performing ANN and validation against a completely new set of data.
• An outline of observations, conclusions, and recommendations regarding the impact of input data quality and ways of mitigating the problem.

3. Results

A rigorous testing procedure, following the development and training of the various ANN models, enabled the assessment of the input data contribution. The graphs included below allow for a visual interpretation of the impact that varying degrees of input quality have on the performance of the ANN, while the accompanying metrics help quantify the same more accurately. For ease of reference, the results are organised in accordance with the structure and nomenclature of the test cases presented earlier herein.

3.1. Impact of Data Quality on ANN Performance

An observation of the following graphs (Figure 7) highlights the excellent fit of the network trained with the full training data (ANN 1). The thick grey line on all graphs represents the ground truth—that is, the actual temperature development on the non-exposed face of the wall through time. The dotted line representing the predictions made by ANN 1 coincides almost entirely with the ground truth line. The above observation is very well recorded and reinforced by the metric results.

The indices used to evaluate the performance of the fully trained network (ANN 1) when tested against the ground truth are summarised in

Table 8. Metric scores for ANN 1 in the testing phase.

Wall Assembly	AME	MAE	RAE	PDIFF	PEP	RMSE	R 2
WA1	1.80	0.56	2.33%	1.80	1.7%	0.75	0.9993
WA2	1.63	0.78	2.78%	1.55	1.4%	0.88	0.9992
WA3	3.03	1.10	4.62%	2.55	2.2%	1.29	0.9981
WA4	2.48	0.83	2.60%	1.39	1.2%	1.05	0.9991
WA5	1.69	0.45	1.42%	−0.65	−0.5%	0.55	0.9998
WA6	1.67	0.64	1.97%	0.70	0.6%	0.78	0.9995
Average	2.05	0.73	2.62%	1.22	1.1%	0.88	0.9992

. The algorithm manages to predict the final temperature with a maximum error of 2.55 °C, which translates to 2.2% (test on Wall Assembly 3) of the scale of the overall developed temperatures throughout the analysis. The good fit is evident not only on the final temperature prediction but throughout the process, where the maximum error appears to be 3.03 °C. The overall agreement between the trained algorithm and the ground truth is demonstrated by the Mean Absolute Error, which on average is 0.73 °C (or a 2.62% difference between the observed values and their mean). An average of 0.9992 coefficient of determination provides robust evidence of the agreement between the observed and predicted data.

The training data offered to ANN 2 was deprived of any middle values of insulation thickness (removed observations incorporating 50 mm insulation externally or internally). The graphical representation of the network’s predictions reveals that the algorithm, despite the reduced data, still captures the “essence” of the physical phenomenon and follows the route of the ground truth curve. Clearly, a perfect fit is not achieved; the prediction lines generally run close but parallel to the ground truth curve. Wall Assembly 4 (WA4) is an exception, where the two lines cross at timestamp 15,300 s; the temperature predictions decline from there onwards.

The above is reflected in the performance indices in

Table 9. Metric scores for ANN 2 in the testing phase.

Wall Assembly	AME	MAE	RAE	PDIFF	PEP	RMSE	R 2
WA1	8.45	2.69	11.3%	6.76	6.27%	3.45	0.9844
WA2	11.44	3.85	13.7%	2.83	2.51%	5.07	0.9741
WA3	32.04	5.06	21.3%	2.34	2.03%	9.74	0.8921
WA4	22.88	6.87	21.6%	11.85	10.37%	8.84	0.9363
WA5	13.09	4.86	15.3%	3.90	2.97%	6.25	0.9702
WA6	9.96	5.73	17.7%	−0.30	−0.24%	6.57	0.9668
Average	16.31	4.85	16.82%	4.56	4.0%	6.65	0.9540

. The predictions of the final temperature are generally within −0.30 °C and 6.76 °C (−0.24% and 6.27% of the observed values, respectively) of the actual values. As observed graphically, the final prediction error for WA4 lies at 11.85 °C (10.37% error compared to the actual temperature value), slightly beyond the range seen on the other tests. Although the prediction and ground truth lines are generally parallel, indicating a good capture of the heat transfer mechanisms, the absolute maximum error of 32.04 °C observed in Wall Assembly 3 is a reminder of the inferior performance of this ANN. This is not an outlier, as on average there is a 16.82% error in all observations made by this network. The degraded performance of this network is also reflected by the lower coefficient of determination and higher root mean square error, which on average are 0.9540 (compared to ANN 1’s score: 0.9992) and 6.65 (compared to ANN 1’s excellent fit score of 0.88), respectively.

A graphical observation of the third network’s (ANN 3) performance reveals similar trends to ANN 2. The predictive capacity of the model is clearly inferior to ANN 1. However, the general mechanism of the heat transfer process has been captured, as indicated by the fact that the ground truth and prediction curves are approximately parallel. The distance between the two lines is larger than observed in the case of ANN 2. It is also worth noting that the curves generated by the ANN 3 predictions appear to be smoother compared to the ones resulting from plotting the predictions of ANN 2.

In a similar fashion, the indices in

Table 10. Metric scores for ANN 3 in the testing phase.

Wall Assembly	AME	MAE	RAE	PDIFF	PEP	RMSE	R2
WA1	28.46	10.87	45.59%	22.58	20.95%	15.05	0.7023
WA2	45.00	16.21	57.80%	45.00	39.93%	22.55	0.4870
WA3	41.09	9.64	40.57%	27.81	24.18%	16.01	0.7081
WA4	48.84	15.12	47.49%	27.71	24.25%	22.35	0.5922
WA5	28.77	11.37	35.85%	14.56	11.07%	15.06	0.8271
WA6	30.34	14.99	46.29%	24.19	19.55%	19.18	0.7171
Average	37.08	13.03	45.60%	26.98	23.3%	18.37	0.6723

reflect the reduced performance of ANN 3. The degradation from the removal of the middle values of the emissivity coefficient appears to be more severe, with absolute maximum error values in the range of 28.46 °C to 48.84 °C. Throughout the six tests, algorithm ANN 3 generates a relative absolute error of 45.60% on average, with a maximum of 57.80% for the values predicted for Wall Assembly 2. The overall performance reduction is reflected by the low average coefficient of determination (R²) and root mean square error (RMSE) of 0.6723 and 18.37, respectively.

The most data-deprived network, ANN 4, presents an irregular graphic form of prediction results. In all tests, there is an approximate agreement between predictions and observed values in the lower range of temperatures. However, when the impact of heat transfer becomes more evident on the finite element analysis model (i.e., ground truth values), ANN 4 fails to react accordingly.

The indices describing the performance of ANN 4 are presented in

Table 11. Metric scores for ANN 4 in the testing phase.

Wall Assembly	AME	MAE	RAE	PDIFF	PEP	RMSE	R2
WA1	18.13	8.36	35.08%	18.13	16.82%	10.18	0.8639
WA2	22.03	10.46	37.30%	14.55	12.92%	12.39	0.8451
WA3	29.39	9.82	41.35%	28.83	25.06%	13.25	0.8002
WA4	33.29	15.44	48.49%	29.33	25.66%	19.49	0.6898
WA5	31.66	8.25	26.01%	31.65	24.06%	12.56	0.8797
WA6	22.70	7.91	24.42%	22.70	18.34%	10.58	0.9140
Average	26.20	10.04	35.44%	24.20	20.5%	13.07	0.8321

. The average coefficient of determination for ANN 4 between the six tests is 0.8321 and the network’s average maximum error is 26.20 °C. ANN 4 generates ultimate temperature predictions that have an error of 14.55 °C to 31.65 °C (12.92% and 24.06% deviation from the actual ultimate temperature). Between the six wall assembly tests, the ANN 4 predictions constitute a mean absolute error of 10.04 °C (35.44% relative absolute error).

Although the above results give an indication of the performance each ANN achieves, depending on the quality and completeness of the offered training data, it is worth presenting the coefficients of determination obtained after training each algorithm. The figures in

Table 12. Final metric scores for each ANN following the completion of their training with their respective training dataset.

Neural Network	Loss
ANN 1	5.0840 × 10−4
ANN 2	5.0721 × 10−4
ANN 3	2.1836 × 10−4
ANN 4	2.4811 × 10−4

indicate the goodness of fit between the predictions made by the algorithms and the corresponding observed values on the test set. They all appear to be performing extremely well; this contrasting behaviour is reflected upon further in the discussion section.

3.2. Performance of the Dominant ANN Model

Following the review of the results presented in the previous paragraphs, the superiority of ANN 1 became apparent. To assess the “dominant” model’s performance against completely unknown data, 6 more tests were carried out. Figure 8 is the visual representation of the results of these additional tests. The goodness of fit between the ground truth (data obtained from finite element model analysis) and the predictions made by the ANN is easy to observe. This is further founded and reinforced by the metrics included in

Table 13. Metric scores for ANN 1, as the dominant network, on unknown data sets (wall test samples).

Test Sample	AME	MAE	RAE	PDIFF	PEP	RMSE	R2
TS1	3.83	0.90	1.89%	−0.44	−0.28%	1.20	0.9995
TS 2	4.26	1.93	3.72%	3.72	2.03%	2.48	0.9981
TS 3	6.06	2.08	6.37%	4.42	3.30%	2.73	0.9946
TS 4	11.42	3.80	26.50%	11.42	15.04%	5.36	0.8990
TS 5	12.43	4.16	10.32%	−4.02	−2.70%	5.81	0.9832
TS 6	4.13	1.92	4.64%	4.03	2.66%	2.25	0.9976
Average	7.02	2.47	8.91%	3.19	3.3%	3.31	0.9787

.

The ANN manages to predict the ultimate temperature developed on the non-exposed surface of the wall test samples with a relatively low error, 3.3% on average. Although on one occasion (Test Sample 4) the prediction of peak temperature differs from the observed value by 11.42 °C (15.04% deviation from the actual temperature), on average, the predictions lie within 3.19 °C of the ground truth. This high predictive performance is observed not only on the peak temperature, as an isolated success, but also by the overall low mean absolute error that ranges from 0.90 °C to 4.16 °C.

Tests TS4 and TS5 appear to have more onerous metric results. The absolute maximum errors are 11.42 °C (in peak temperature, as explained above) and 12.43 °C, respectively. These can be observed in the graphic representation of the results as deviations of the prediction curve from the ground truth curve. Despite these local inconsistencies with the observed figures, the overall high coefficient of determination (0.9787 on average between tests) and low root mean squared error (3.31 on average) indicate a high-performing model.

4. Discussion

Although an algorithm capable of predicting the temperature developed on the wall samples’ non-exposed face was eventually constructed, a few items worth highlighting and discussing further appeared during the development and evaluation process. These are listed in the following paragraphs, with the intention to evoke thoughts and discussion regarding potential pitfalls and respective solutions when the ANNs are employed on heat transfer through masonry wall applications.

The perfect fit of ANN 1 with the ground truth is not a surprising result. The network was trained with the full training data, which means that its comparison with the ground truth merely verifies whether its predictions can replicate already-known patterns and figures. Despite its obvious results, however, this step is far from unnecessary, as it ensures the replicative validity of the developed algorithm. It helps in building confidence that the network is not only functional but that it is also able to identify and capture the underlying patterns in the provided dataset.

After achieving this first level of validation, the research could proceed by exploring the limits of the algorithm architecture by varying the quality and quantity of the provided training data. ANN 2, deprived of mid-range insulation thickness values, and ANN 3, deprived of mid-range emissivity coefficient values, both had difficulty in achieving equally high predictive rates as ANN 1, which was trained with the full range of data. It is worth highlighting that the predictions of ANN 2 lay closer to the ground truth but demonstrated some local irregularities. On the contrary, the predictions generated by ANN 3 lay further from the ground truth curve (as reflected by the more onerous performance indices); however, these were largely offset from the observed figures, following their smooth curvature. This raises questions as to the impact different variables may have on the performance of ANNs, depending on their contribution to the physical phenomenon under consideration. The emissivity coefficient is a constant value throughout the finite element analysis, while properties relating to the EPS are time-dependent (within the context of this study). The physical importance of the independent variables needs to be well understood before they are incorporated into an ANN structure.

The most data-deprived network, ANN 4, presents the most irregular graphic form of prediction results. The comparison between the performance indices for ANN 4 and ANN 3 show that the former performed better. However, a visual observation shows clearly that the prediction line of ANN 4 is more erratic compared to ANN 3, whose line is just “offset” (i.e., consistently overestimating or underestimating compared to the ground truth). Although networks ANN 2 and ANN 3 failed to make accurate predictions (at least to the degree ANN 1 did), they appeared to capture the underlying principles and function of the physical phenomenon. On the other hand, the lack of resemblance to the ground truth curve demonstrates that the capture of the underlying mechanisms of the physical phenomenon by ANN 4 is poor, despite its better metrics.

At the training stage, all algorithms returned high indices of fitness to the test data. However, their performance on predicting values beyond their training sets varied greatly depending on the amount and quality of data that was offered at that initial stage. This underlines the need for the validation of the algorithms and a multifaceted evaluation of their performance prior to their application in further research or field operations. It also shows that, despite developing a functioning and potentially effective ANN architecture, the ultimate performance might be compromised by a lack of representative observations in the training set.

5. Conclusions and Further Research

This paper contributes towards the further integration of ANNs in the field of heat transfer through building materials and assemblies. Arguably, there is a wide scope for further development of this research in directions such as the use of different algorithm types and structures, an extended range of building assemblies and materials, and different thermal loadings. Nevertheless, some first conclusions can be drawn from this initial effort, informing future scientific work aiming to employ Artificial Neural Networks for the description of heat transfer phenomena. Although good metric results quantifying the replicative validity of the algorithm offer an indication of a functional ANN, they do not necessarily constitute evidence of a fully operational and reliable network. As such, the use of further validation techniques is of paramount importance. A comparison against unknown data (methodology followed in this study) is an objective way of ensuring the constructed model behaves and performs as expected. When “external” data is not available or difficult to obtain, k-fold cross validation could provide a route for building some confidence in the performance of the model. To mitigate the risk of overfitting, neuron “dropout” can be introduced as part of the algorithm’s hyperparameters. This would artificially weaken the fitting of each neuron to the supplied training data.

As seen in the previous paragraphs, metrics and indices can sometimes be misleading. An intuitive understanding of the predictions and their relevance to factual data is necessary to mitigate the impact of the “black box” phenomenon of blindly accepting output generated by an AI model. Outlining the expectations of the models’ output in advance could provide a measure against which a preliminary assessment of the ANN’s predictions can be undertaken. In the same direction, producing visual representations of the output at every stage can enhance the understanding of the relationship between the ground truth and the predictions and can imminently highlight subtle inaccuracies or major errors.

Depending on the nature and contribution of each parameter to the physical phenomenon, the impact of its loss from the training data can have a more severe impact on the predictive capability of the Neural Network. It is worth researching this relationship by quantifying the contribution of various parameters on the development of a physical phenomenon and then training ANNs with a gradual deprivation of the variables in question. This could provide an opportunity for quantifying the correlation of various physical parameters with the performance of the artificial neural model.

The ANN’s development, analysis, and interrogation need to be considered within the wider context of a specific scientific study or practical application. This contextualisation can have a significant effect on the specification of required accuracy levels for the chosen evaluation methodology. Different scientific applications have different required levels of accuracy in their produced results. As such, it is prudent to avoid trying to establish ill-defined accuracy thresholds without specific application requirements to hand.

This study presented the underpinning principles of an assessment methodology for the evaluation of ANN predictions and highlighted potential pitfalls arising from the use of ANNs within a masonry wall heat transfer context. The focus of the present work is to identify the impact of varying quantities and qualities of input data on the accuracy of a specific ANN architecture and to propose a methodology for demonstrating this relationship. This objective has been accomplished by presenting the inferior results generated by the models trained with reduced amounts of data. Ultimately, the proposed methodology for the assessment and validation of the ANN’s performance was proposed not as a panacea for all ML evaluation problems, or as an optimum precision benchmark, but as a first step towards preventing (or at least enabling the quantification of) accuracy deficiencies in models developed using the data each research team has available.

It is hoped by the authors that future work can feed into the existing proposed protocols for the development of ANNs while aligning such documentation to the needs of heat transfer and building fire research. Expanding the existing understanding of the factors impacting the performance of ANNs and incorporating elements specific to fire engineering and heat transfer through building elements could help safeguard future research in this field from misleading results or discrepancies caused by different ANN models and parameters.