3.1. Analysis of Dataset
The quality of training data has a huge impact on the efficiency, accuracy, and complexity of DNN models. Compiled raw datasets may contain data points from selected features that are physically not relevant for the studied problem. In the particular dataset presented above, several data points were found to be characterised by a scaled distance
, which lies out of the design range considered for contact and close-in detonations, according to UFC 3-340-02 [
3]. As such, all data points with an associated scaled distance
were removed from the dataset. Raw data also include data points that are significantly different from the rest of the data, the so-called outliers. Ideally, data points should be statistically relevant to the output data that a model intends to predict. Therefore, it is important to properly filter the dataset to ensure that the observations best represent the physical problem. Generally, outliers should be carefully hand-picked to ensure the best expected performance, but this process is generally time-consuming, particularly when analysing large sets of data. A commonly used approach to detect and remove outliers is the Inter-Quartile Range (IQR) score, which is a measure of statistical dispersion, defined as
where Q3 and Q1 are the 75
th and the 25
th percentiles of the data, respectively. The IQR score is used to define lower and upper bounds, where any data point lying outside this range is considered an outlier and, consequently, removed from the dataset. This decision range is given by
where
k is the scale factor. In this case, the scale factor is taken as
, as it represents the factor that induces the filtered data to be within the range of three standard deviations (
) from the mean
in an equivalent Gaussian distribution, thereby likely avoiding the deletion of pertinent data. Given the characteristics of the dataset, a filter is only applied on the upper bound of the data distribution.
Table 1 lists the count, range/categories, average and standard deviation of each input feature after filtering, while
Figure 3 depicts the distributions of the continuous features. With a data range between two extreme bounds, performance prediction using the current dataset would lead to a more accurate result, where data are more abundant within those bounds (i.e., high density).
Additionally, the relevance of each input parameter is evaluated by means of a comprehensive statistical analysis using the Pearson Correlation Coefficient (
r). It expresses the strength of the relationship between two variables and is defined as
where
n is the sample size, and
X and
Y are the two variables to be correlated. Positive correlation occurs when two variables evolve in the same direction (i.e., one variable increases/decreases its value with the other), while negative correlation represents an opposite evolution of two variables. The smaller the absolute value of this coefficient, the lower the dependency observed between two specific variables. As a rule of thumb, it is accepted that a coefficient ranging from 0.7 to 1.0 (−0.70 to −1.0) indicates high correlation between variables, moderate correlation from 0.5 to 0.7 (−0.5 to −0.7), low correlation from 0.3 to 0.5 (−0.3 to −0.5), and negligible correlation from 0.0 to 0.3 (0.0 to −0.3) [
34].
Figure 4 shows the correlation coefficients for the interaction of all input features. As expected, input features that are physically not correlated present very small correlation factors (e.g., concrete compressive strength and charge weight), while features that are known to influence each other present larger coefficients [
35]. It should be noted that although the input parameters are assumed to be independent variables for the purpose of modelling, some variables are intrinsically related (e.g., charge weight and charge diameter). An ideal condition would require calculating the network connectivity with corresponding weights.
A similar relationship between input–output variables can be obtained by performing a one-way analysis of variance (ANOVA). This identifies the relevance of each input feature to the performance of the targets by using the F-statistic and
p-value as the relevance metrics. The
p-value ranges from 0 to 1 and estimates the probability of failing to reject the null hypothesis. As such, the smaller the
p-value, the smaller the probability. On the other hand, the F-statistic reflects the implication of the regression coefficient. The higher this value and the smaller the value of
p, the more statistical significance the corresponding input feature has to the correlated output. In this work, a confidence interval of the ANOVA is set to 95% (
p-value < 0.05) to reject the null hypothesis. Thus, input features with a calculated
p-value less than 0.05 are considered substantial [
36]. The calculated F-statistic and
p-value for every input feature are listed in
Table 2. From the data, it is visible that while all features included in the dataset are considered relevant in determining the output data, features with higher relevance include the charge shape, target thickness, concrete compressive strength, and charge diameter. Interestingly, the standoff distance is considered the least relevant in this list of features, most probably due to the limited range included in the database. Following these results, all features are considered in the training process of the neural network model.
In addition to data cleansing, a normalisation technique is applied to all input features in the dataset. It seeks to make different samples seen by a neural network model more similar to each other, which helps the model in generalising well to new data [
16]. The most common form of data normalisation involves centring the data around zero, through the z-score method, as calculated by
where
Y denotes the normalised data and
X,
, and
indicate the original data, their average, and their standard deviation, respectively.
3.2. Network Architecture
The behaviour of a neural network model is controlled by a set of constants, referred to as structural hyperparameters, such as neurons per hidden layer, number of hidden layers, and learning rate. Hyperparameters are adjustable parameters that let you control the training process and heavily affect the model performance. Although several approaches to determining the number of hidden layers and neurons per hidden layer have been used, no general guideline has been formulated [
37,
38]. As such, the number of neurons and hidden layers were optimised by the improvement (tendency) in the accuracy of prediction through an independent two-step process. The initial number of neurons per hidden layer was set to be 512, and the number of hidden layers was systematically increased from one to ten. For every iteration, the accuracy of prediction was registered for up to 500 epochs. Similarly, the optimisation of the number of neurons per hidden layer was performed by setting each hidden layer to have 64 initial neurons with a subsequent two-fold at each iteration. For every iteration, the model was trained on a 80/20 training/validation split subset. The division of the entire dataset into the training set and test set was randomly carried out, and the performance of the model on the test set was an indication of the performance of the model on the unknown data.
Throughout the training process, the neural network receives input data and generates output predictions, which are subsequently evaluated against the real target values. The dissimilarity between the projected output and the true target value is referred to as the loss or cost function. To find the optimal set of parameters that minimises this function, the Adaptive Moment Estimation (Adam) optimisation algorithm was implemented to efficiently search for the optimal set of parameters, such as learning rate, momentum, decay rates, and batch sizes [
39]. These parameters determine how the algorithm updates the weights and biases of the neural network during training.
To prevent overfitting, which occurs when a neural network performs well on training data but is unable to generalise and perform accurately on unseen data, weight regularisation was implemented using the L2 regularisation method [
40]. This method adds penalty factors to the network layers to alter the weight propagation through the layers, facilitating the model to converge optimally. Finally, the number of neurons on the last hidden layers was also analysed and reduced across the network, with a dropout rate of 25%.
Overall, the input layer contains the features listed in
Table 1, while the output layer is based on the performance categories (no damage, spall, and breach). The model predicts the probability of each performance category, selecting the one with the highest probability of occurrence as the assumed output performance.
Table 3 shows an example of the output category selection process for random data points from the testing dataset.
Figure 5 depicts the variation in prediction accuracy (for training and validation subsets) for an increasing number of hidden layers and neurons at the last epoch. To achieve a stable level of accuracy on the predicted data, the final neural network model was set to be built with seven hidden layers and 512 neurons per layer. The architecture of the DNN model is shown in
Figure 6. The tuned parameters are listed in
Table 4.