Using Bootstrapping to Determine Artificial Neural Network Confidence Intervals—Case Study of Particleboard Internal Bond Determined from Production Data

Abstract

Tensile strength perpendicular to the plane of the board (also known as the Internal Bond—IB), determined in accordance with standard EN 319, is one of the most critical properties in particleboard quality control. Given the need for efficient, rapid methods to assess the IB in industrial contexts, artificial neural networks (ANN) have been used as a predictive modelling tool. However, one of the main limitations of these techniques is the absence of estimates associated with the uncertainty of their predictions. The present study addresses this shortfall by applying bootstrap techniques to obtain confidence intervals using estimates generated by ANN. To achieve this, multiple models were trained and validated using experimental data taken from real production processes. The results show that the methodology proposed can be used to obtain a high level of accuracy (determination coefficient R² = 0.96) and a coverage probability of 93%. It also provides a robust criterion to assess conformity with standard specifications. This study concludes that adding bootstrap to ANN modelling is a very useful tool for application in industrial quality control systems, as it allows decision making based on confidence intervals rather than individual values.

1. Introduction

European Standard EN 309 [1] states that a particleboard is a wood-based product manufactured by applying pressure and heat to particles of wood or other lignocellulosic materials glued with different types of adhesive depending on their final destination.

Global particleboard production grew slightly until the early 2000s, stabilising at around 100 million cubic metres a year and generating a market of around 14.2 × 10³ million USD [2].

One of the most important properties assessed in a particleboard is the internal bond because it is an indication of correct bonding and pressing. This property is measured by determining the tensile strength perpendicular to the plane of the board, following EN 319 [3]. However, the correct manufacture is affected by many parameters that influence the final performance of a particleboard [4,5,6]. Major advances in electronics in recent years have enhanced the ability to collect a large amount of reliable data from industrial processes. Along with the increase in computational power in the last 10 years, these advances have led to the development of powerful tools based on artificial intelligence that can be applied to assess how production parameters influence particleboard properties and optimise industrial processes. Process optimisation, based on statistical studies of industrial processes, can be approached in two ways. The first is to use descriptive statistical techniques, such as Shewart control charts [7,8]. The second is to apply modelling techniques based on multivariate regressions or artificial neural networks (ANN) to model a mechanical property using production parameters or physical properties, obtaining a mathematical model for the subsequent optimisation of variables [6,9,10,11,12,13,14,15,16,17]. Some studies have even combined neural network modelling techniques and Shewart control charts [18,19,20,21].

While these techniques, particularly those based on ANN, provide models with high reliability (determination coefficients higher than 70%) [13,14,22], they lack necessary information about the accuracy of the output of the modelling, which is typically expressed as confidence intervals or uncertainty. This uncertainty is a result of the inability to determine an optimum set of independent variables to model a process, and the uncertainties associated with the variables when they are measured. Moreover, the simplifications, insufficiencies, and ambiguity inherent to representing real-world processes by mathematical equations add to the structural uncertainty of the final models. This is a significant limitation on the efficacy of ANN, particularly in decision-making tasks, where it can compromise the reproducibility and reliability of the results of a model [23].

The measure of accuracy, expressed as the standard deviation or the confidence intervals of the estimator, is one of the most important aspects of the statistic process of modelling [24] because it provides very precise indications of the reliability of the modelling process and supports future decision making based on the estimator [25,26]. Similarly, production control cannot be based solely on obtaining the data of a property alongside traditional statistical measures such as variance or mode. Other estimates are required, including the confidence intervals of the control variable [27], because without these estimates, any specification compliance report based on the models developed would not be completely reliable [28,29].

Several authors have obtained the confidence intervals of the control variable at an industry level, using both production data and simulation data. Young et al. [27] evaluated three bootstrap methods for obtaining confidence intervals (a fully nonparametric, fully parametric, and nonparametric bootstrap for parametric inference (NBSP)) to analyse the modulus of elasticity (MOE) and the modulus of rupture (MOR) in wood–plastic composites (WPC) using test data. They concluded that, although one method may be preferable to another depending on the circumstances, obtaining the confidence intervals can help to improve the characterisation of the properties of a material and aid early failure detection. In a later study, also using test data, these authors noted the importance of obtaining the confidence intervals, but in this case, they applied Bayesian methods for the estimate [30]. Edwards et al. [31] noted the importance of estimating confidence intervals in medium-density fibreboard (MDF) production processes, reporting that bootstrapping provides an efficient solution to estimate confidence intervals when other approaches are inefficient or less realistic. They used 2000 resamplings for the bootstrapping, less than half the number applied in the present article, to calculate the lower percentiles of the internal bond from board manufacturing data.

Khosravi et al. [32] used data from various fields, including mathematics, medicine, transport, environmental studies, electronics, and production processes, and trained the neural network so that instead of providing a simulation of the result, it would provide the confidence intervals of the result, using two neurons in the output layer. Khosravi et al. [33] also conducted a comparative study of various techniques used to obtain confidence intervals from neural network results and the delta, Bayesian, bootstrap, and mean–variance estimation (MVE) methods. They concluded that bootstrapping is a powerful tool for constructing confidence intervals that balances current computational power with the accuracy of the result.

Similarly, Khosravi et al. [34] reaffirmed the bootstrap technique as a powerful tool for generating confidence intervals. In this study, they combined artificial neural networks and bootstrap methods to obtain confidence intervals using the estimate of variance and real-world and simulated data to assess the reliability of the method.

García Fernández et al. [35] addressed the need to know the confidence intervals of a neural network prediction, and the problem of neural networks being regarded as some sort of black box. They addressed the problem using a Monte Carlo simulation, a slightly different resampling technique from bootstrapping, and concluded that Monte Carlo is also a useful method to obtain confidence intervals.

Hosen et al. [36,37] proposed a combination of neural networks to generate prediction confidence intervals, using a weighted average of each network in the final result and improving the quality of the confidence intervals with regard to the accuracy and the coverage index.

The objective of this study was to use bootstrapping to determine the confidence intervals of the prediction of a standard particleboard internal bond obtained by artificial intelligence techniques, enabling greater reliability in factory decision making about specification compliance. Following Nerea Portillo et al. [38] and Stoma et al. [39], bootstrapping is the most appropriate method to address input data uncertainty because it enables an optimum performance while also minimising computation costs. Moreover, bootstrapping is relatively simple compared to other methods of constructing prediction intervals (PI), such as the delta [40] or Bayesian [41] techniques. As bootstrap-generated PI tend to be narrower, they generate better information about the result. Unlike alternative approaches, bootstrapping does not require a calculation of complex matrices such as the Jacobian or Hessian matrices. This improves the stability of the method and avoids problems associated with mathematical singularities [25,34]. To this end, the internal bond was calculated by combining the outputs of 20 artificial neural networks to improve the accuracy of the overall result.

2. Materials and Methods

Particleboard is a highly industrialised wood-based product. It was first mentioned in 1887, but until 1910, it was not possible to obtain suitable wood flakes for production [5]. Particleboard is manufactured mainly from debarked round wood that has been chipped in a wood chipper. After drying, the flakes, or particles, are put through a gluing machine, normally horizontal, where they are sprayed with adhesive to evenly coat their entire surface. Once the particles have been glued, they are dosed through a spreading machine on a mat assembly belt, and the final board is obtained from the mat in a press.

Material comprised 300 panels of particleboard type P2 taken at random from the production line during one production year. Following EN 312 [42], these panels are made using urea formaldehyde resin and are intended mostly for the manufacture of indoor furniture. The most commonly used bonding quality test is internal bond (Figure 1) or tensile strength perpendicular to the plane of the board. Panels were cut following standard EN 326-1 [43] and tested for internal bond (N/mm²) as per EN 319 [3], moisture content of the board at the time of testing (%) as per EN 322 [44], and density of the board at the time of testing (kg/m³) as per EN 323 [45] in the factory production control laboratory. The internal bond test consists of determining internal bond strength of a board by tensile strength perpendicular to the plane. It is crucial in production control, as it indicates how the gluing and pressing process is proceeding.

Other production line data were collected at the same time from the production line sensors: particle temperature (°C), measured after drying; resin temperature (°C), measured when the flakes entered the gluing machine; mat moisture content (%), measured after drying; mat speed (mm/s); resin percentage (%), measured from board weight; additive (paraffin) percentage (%); and mean press temperature (°C) during the process.

2.1. Bootstrap Methodology

Bootstrapping is a resampling technique used to determine the parameters of an unknown distribution of an estimator through an empirical distribution [46]. The idea underlying bootstrapping is that the empirical distribution is a good approximation of the theoretical distribution of the estimator [31]. The empirical distribution can be used to calculate the error, bias, percentiles, and confidence intervals of the estimator. In this study, BCa (Bias-Corrected and accelerated) bootstrapping is applied. This produces good confidence intervals based on the percentiles of the bootstrap distribution of the estimator. Described by Efron and Tibshirani [46], this methodology has been used to obtain robust confidence intervals for various properties of wood-based products [27,30,31].

When applying bootstrapping to ANN modelling, we start from n ANN developed from various training, validation, and testing sets obtained at random. The modelling output will be the mean of n outputs [32].

Let $\mathrm{x}=\left({\mathrm{x}}_1,{\mathrm{x}}_2,\dots .{\mathrm{x}}_{\mathrm{n}}\right)$ be the vector formed by the outputs of the n networks developed. Let the bootstrap sample ${\mathrm{x}}^{*\mathrm{b}}=\left({\mathrm{x}}_1^{*\mathrm{b}},{\mathrm{x}}_2^{*\mathrm{b}},...{\mathrm{x}}_{\mathrm{n}}^{*\mathrm{b}}\right)$ be the sample obtained through random replacement sampling B number of times. And let ($\widehat{\mathrm{\Theta }}$) be the real distribution of the estimator we want to obtain from the sample, in this case, the mean.

Using the mean of every bth bootstrap sample (${\widehat{\mathrm{\Theta }}}^{*\mathrm{b}}$), the empirical distribution of the bootstrap estimator for the mean is calculated (${\widehat{\mathrm{\Theta }}}^{*}$). Following Efron and Tibshirani [38] the distribution (${\widehat{\mathrm{\Theta }}}^{*}$) can be used as the estimator of the real distribution ($\widehat{\mathrm{\Theta }}$) of the mean.

One of the major advantages of this resampling method is that is not necessary to assume any hypotheses about the underlying distribution ($\widehat{\mathrm{\Theta }}$) of the statistic studied, in this case, the mean [27].

Of all the methods for determining confidence intervals described by Efron and Tibshirani [46], BCa is the only one that is second-order accurate and transformation respecting and produces better results because they are closest to the real confidence intervals.

The BCa method is based on the percentiles of the bootstrap distribution (${\widehat{\mathrm{\Theta }}}^{*}$), but the percentiles are corrected using two parameters ($\widehat{\alpha }$ and ${\widehat{\mathrm{z}}}_0$) known as acceleration and bias correction (Equation (1)).

$${\widehat{\mathrm{z}}}_0={\mathrm{\varphi }}^{-1}\left({\displaystyle \frac{\#\left({\widehat{\mathrm{\Theta }}}^{*\mathrm{b}}<\widehat{\mathrm{\Theta }}\right)}{\mathrm{B}}}\right)$$

(1)

where ${\mathrm{\varphi }}^{-1}$ is the inverse function of a standard normal cumulative distribution function. In other words, ${\widehat{\mathrm{z}}}_0$ measures the bias of the median of ${\widehat{\mathrm{\Theta }}}^{*}$; i.e., the discrepancy between the median of ${\widehat{\mathrm{\Theta }}}^{*}$ and $\widehat{\mathrm{\Theta }}$.

Therefore, a BCa confidence interval with coverage of 100(1 − α) percent for our estimator $\widehat{\mathrm{\Theta }}$ will have the form (Equation (2)):

$$\left[{\widehat{\mathrm{\Theta }}}^{*\left({\alpha }_1\right)},{\widehat{\mathrm{\Theta }}}^{*\left({\alpha }_2\right)}\right]$$

(2)

where ${\alpha }_1$ and ${\alpha }_2$ are the acceleration (Equation (3)):

$$\begin{array}{c}{\alpha }_1=\mathrm{\varphi }\left({2\widehat{\mathrm{z}}}_0+{\mathrm{z}}^{\left({\displaystyle \raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\right)}\right)\\ {\alpha }_2=\mathrm{\varphi }\left({2\widehat{\mathrm{z}}}_0+{\mathrm{z}}^{\left(1-{\displaystyle \raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\right)}\right)\end{array}$$

(3)

where $\mathrm{\varphi }$ is the standard normal cumulative function and ${\mathrm{z}}^{\left(\alpha \right)}$ is the 100αth percentile of the standard normal distribution.

To assess the quality of the confidence intervals, the prediction interval coverage probability (PICP) was used. This indicator has been widely applied to determine confidence interval quality [25,33,47,48,49,50,51,52] (Equation (4)). It assesses the number of experimental data included in the confidence interval. Following Khosravi et al. [33], PICP is a good indicator of the quality of the confidence intervals obtained.

$$\begin{array}{c}\mathrm{P}\mathrm{I}\mathrm{C}\mathrm{P}\left(\%\right)=100·{\displaystyle \frac{1}{{\mathrm{n}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}}}{\displaystyle \sum _1^{{\mathrm{n}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}}}{\mathrm{c}}_{\mathrm{i}}\\ \\ {\mathrm{c}}_{\mathrm{i}}=\left\{\begin{array}{c}1 \mathrm{s}\mathrm{i} {\mathrm{t}}_{\mathrm{i}}\in \left[{\mathrm{L}}_{\mathrm{i}},{\mathrm{U}}_{\mathrm{i}}\right]\\ 0 \mathrm{s}\mathrm{i} {\mathrm{t}}_{\mathrm{i}}\notin \left({\mathrm{L}}_{\mathrm{i}},{\mathrm{U}}_{\mathrm{i}}\right)\end{array}\right\}\end{array}$$

(4)

where ${\mathrm{n}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ is the number of experimental data, t_i is the i-th experimental value, and L_i and U_i are the upper and lower limits of the confidence intervals of the i-th value.

2.2. Artificial Neural Networks (ANN)

The nature of some ANNs, such as the multilayer perceptron, as universal approximators (Figure 2) [53], and their robustness against outliers, give them the ability to model all kinds of relations between variables with a high degree of reliability, especially when obtaining an accurate result is more important than knowing the relationships between the variables involved in the process [54].

The ANN model chosen was the multilayer perceptron. This network type has demonstrated its ability to model various properties of wood-based products, particularly in boards [12,13,14,55,56], but also in solid wood [57] and other products [16,58,59].

The network input variables were the temperature of particles and resin, mat moisture content, resin percentage, percentage of additives, mat speed, and press temperature. The output variable was internal bond.

The type of network chosen to develop the internal structure was the multilayer perceptron, as described by the authors elsewhere [12,13,60]. As there is no precise rule to define the internal structure of a network, it was determined in each case by trial and error. To avoid the problem of overfitting, the initial dataset was randomly divided into three subsets (training, validation, and testing) at percentages of 60, 20, and 20% of the total dataset, with the condition of there being no significant differences between the subsets for a confidence level of 95%. In this way, the representativeness of the datasets in respect to the total dataset was verified. The training subset was used for the learning of the model, while the validation subset was used to monitor and prevent over-fitting during training. The test subset was used to independently assess the reliability of the model. Training was controlled by the early-stopping technique. This stops the process when a 10% variation occurs in the validation set in two consecutive cycles, either in the mean squared error (mse) or in the correlation coefficient.

Before the network was designed, a principal component analysis was conducted on the input vector to detect any correlation between the input variables and improve the network output [61]. The selection criterion for the variables was that they had to contribute more than 2% to sample variability [61,62].

All data were normalised (Equation (5)) before they were used to develop the networks, to improve network performance [61].

$${\mathrm{X}}^{\prime }={\displaystyle \frac{\mathrm{X}-{\mathrm{X}}_{\mathrm{m}}}{\mathrm{s}}}$$

(5)

where X′ is the value after vector X has been normalised, X_m is the mean of vector X, and S is the standard deviation of vector X.

In all the networks, the transfer function used was the hyperbolic tangent sigmoid (Equation (6)), to enhance network performance [61].

$$\mathrm{f}\left(\mathrm{x}\right)={\displaystyle \frac{2}{1+{\mathrm{e}}^{\left(-2\mathrm{x}\right)}}}-1$$

(6)

where f(x) is the output value of the neuron and x is the input value of the neuron.

All the networks were developed so that they would be mathematically defined [63]; i.e., the training set had more data than the number of internal connections in the network.

The acceptance criterion for the networks was defined as the absence of significant differences between the real data and the simulators for the testing set, following the 2014 version of standard EN 326-2 [64]. For greater assurance in the modelling output, the criterion applied was a correlation coefficient (R) greater than 0.70 between the real data and the simulated data, following the requirements of the 2000 version of standard EN 326-2 [65] to accept test results obtained using non-standard test methods.

To calculate the confidence intervals, 250 different ANN were initially developed, varying the number of hidden layers and the number of neurons in each layer. Twenty of these ANN were selected as having the highest number of correlation coefficients in the testing set (Figure 3) and complied with the criterion of standards EN 326-2 [64,65]. The number of networks chosen agrees with that used by Papadopoulus, Edwards and Murray [66], and Secchi, Zoi, and di Maio [67], who included 20 and 15 networks, respectively, in their simulations, and exceeds the number of networks used by Khosravi et al. [25] and Khosravi et al. [32], who used only 10 different ANN models in their research, and Stoma et al. [39], who used 10 data items for their simulation.

The empirical distributions of each output vector were calculated from 5000 datasets obtained by random resampling with replacement. The intervals were obtained using a significance index of 95%, which is commonly applied to obtain the confidence intervals of particleboard test results.

3. Results

The characteristics of the particleboards manufactured are shown in

Table 1. Particleboard characteristics. MC: moisture content.

Property	Mean	Std. Deviation	Min.	Max.
Thickness (mm)	19.80	7.38	7.82	40.14
Density (kg/m3)	668.81	37.09	562.41	782.78
MC (%)	9.48	0.73	7.68	14.30

.

The values of the production parameters obtained in real time and the value of the internal bond determined in the laboratory are shown in

Table 2. Production and internal bond parameters.

Variable	Mean	Std. Deviation	Min	Max
Particle temperature (°C)	43.76	6.21	30	57
Resin temperature (°C)	27.9	10.3	7.5	40.4
Mat moisture content (%)	4.7	0.6	6.6	3.6
Resin percentage (%)	8.90	0.27	8.15	9.75
Additive percentage (%)	0.30	0.03	0.16	0.48
Mat velocity (mm/s)	455.8	119.7	185	755
Press temperature (°C)	230.4	9.8	184.4	268.4
Internal bond (N/mm2)	0.56	0.11	0.30	0.98

.

As the principal component analysis showed that each variable contributed more than 2% to variability, the variables were not transformed.

3.1. Artificial Neural Network (ANN) Results

Table 3. Results of the artificial neural network design process. R²: determination coefficient, R: correlation coefficient, p-value: statistic to identify significant differences between simulated and experimental values.

Network	Phase	Structure	R2	R	p-Value
1	Training	[7 3 3]	0.82	0.91	0.96
	Validation		0.80	0.89	0.82
	Testing		0.57	0.75	0.72
	Total		0.76	0.87	0.76
2	Training	[8 1]	0.81	0.90	0.95
	Validation		0.73	0.85	0.29
	Testing		0.55	0.73	0.97
	Total		0.71	0.84	0.70
3	Training	[8 2]	0.71	0.84	0.87
	Validation		0.69	0.83	0.24
	Testing		0.60	0.77	0.93
	Total		0.66	0.81	0.72
4	Training	[7 1 1]	0.82	0.91	0.59
	Validation		0.69	0.83	0.72
	Testing		0.63	0.79	0.39
	Total		0.74	0.86	0.50
5	Training	[7 1 1]	0.78	0.88	0.92
	Validation		0.69	0.83	0.41
	Testing		0.50	0.71	0.72
	Total		0.69	0.83	0.61
6	Training	[5 2 2]	0.68	0.83	0.76
	Validation		0.67	0.82	0.24
	Testing		0.63	0.79	0.47
	Total		0.62	0.79	0.61
7	Training	[8 1]	0.69	0.83	0.20
	Validation		0.66	0.81	0.67
	Testing		0.64	0.80	0.99
	Total		0.64	0.80	0.46
8	Training	[7 2 1]	0.86	0.93	0.65
	Validation		0.66	0.81	0.51
	Testing		0.56	0.75	0.21
	Total		0.73	0.85	0.25
9	Training	[8 2]	0.82	0.90	0.95
	Validation		0.64	0.80	0.51
	Testing		0.51	0.72	0.92
	Total		0.72	0.85	0.89
10	Training	[5 1 1]	0.80	0.89	0.95
	Validation		0.63	0.80	0.14
	Testing		0.63	0.79	0.85
	Total		0.69	0.83	0.51
11	Training	[5 1]	0.70	0.84	0.95
	Validation		0.64	0.80	0.35
	Testing		0.66	0.81	0.60
	Total		0.66	0.81	0.67
12	Training	[7 2 1]	0.90	0.95	0.98
	Validation		0.62	0.79	0.18
	Testing		0.59	0.77	0.40
	Total		0.73	0.86	0.34
13	Training	[5 1 1]	0.79	0.89	0.98
	Validation		0.62	0.79	0.22
	Testing		0.51	0.71	0.65
	Total		0.67	0.82	0.74
14	Training	[4 3 3]	0.85	0.92	0.99
	Validation		0.63	0.79	0.71
	Testing		0.54	0.73	0.64
	Total		0.74	0.86	0.72
15	Training	[7 2 2]	0.71	0.84	0.98
	Validation		0.61	0.78	0.43
	Testing		0.62	0.79	0.21
	Total		0.66	0.81	0.42
16	Training	[7 2]	0.78	0.89	0.93
	Validation		0.60	0.78	0.19
	Testing		0.60	0.78	0.90
	Total		0.68	0.82	0.61
17	Training	[5 3 3]	0.83	0.91	0.99
	Validation		0.60	0.78	0.23
	Testing		0.58	0.76	0.89
	Total		0.71	0.84	0.58
18	Training	[5 2 2]	0.76	0.87	0.98
	Validation		0.61	0.78	0.29
	Testing		0.57	0.75	0.35
	Total		0.66	0.82	0.39
19	Training	[5 1]	0.75	0.87	0.95
	Validation		0.59	0.77	0.27
	Testing		0.50	0.71	0.97
	Total		0.65	0.80	0.68
20	Training	[7 1]	0.63	0.80	0.89
	Validation		0.58	0.76	0.68
	Testing		0.60	0.77	0.21
	Total		0.61	0.78	0.44

Note 1: the ‘Structure’ column displays only the number of neurons in the hidden layers.

shows the results of the training, validation, and testing processes for the internal bond test.

For the testing sets, the value of the determination coefficients (R²) obtained shows that the models developed can explain at least 50% of sample variability.

The correlation coefficients in the testing set (0.71 to 0.81) exceed the minimum value of 0.70 indicated in the 2000 version of standard EN 326-2 [65] to accept the test results obtained using a non-standard method.

The p-value exceeds 0.05 in all cases, and therefore, there are no significant differences between the test values and the ANN outputs for a significance level of 95%, as specified in the 2014 version of standard EN 326-2 [64]. This means that all the conditions have been met for the acceptance of the ANNs developed.

3.2. Bootstrap Results

Table 4. Results of the regression between the mean value of the 20 networks and the real value.

Property	Equation	R2	R	RMSE	p-Value	PICP (%)
Internal bond	0.986·X − 0.003	0.96	0.98	0.06	0.97	93.0

shows the correlation and the root mean square error for all the data between the original values and the mean of the simulations by the 20 chosen networks.

The determination coefficient obtained (R²) indicates that bootstrapping neural networks is capable of explaining at least 96% of the variability in the sample. In addition, the correlation coefficient obtained (R) exceeds the threshold established in standard EN 326-2 [65] for accepting an alternative method to the standard procedure. This standard states that, for an alternative method to be valid, a minimum correlation coefficient of 0.70 must be obtained, otherwise the method is not acceptable.

In addition, the p-value obtained is greater than 0.05, and thus it can be concluded that there are no statistically significant differences between the experimental results and the result obtained by bootstrapping the neural networks, applying a confidence level of 95%, following the criteria established in standard EN 326-2 [64].

Therefore, compliance of all the requirements to consider the bootstrap ensemble valid and acceptable is verified from the results obtained using the neural networks developed in this study.

The PICP value obtained (93%) indicates that almost all the experimental data are within the confidence intervals generated.

Figure 4 shows the relation between the experimental values and the mean of the outputs of each network for the total dataset.

4. Discussion

As initially stated, the main problem of ANN-based modelling is that, although it can provide accurate predictions, it provides no information about the uncertainty of the predictions. In the context of industrial production, it is not sufficient to obtain an estimated value of a mechanical property such as the internal bond. The confidence interval of the estimate must also be known, to ensure that both the mean and its margin of variability comply with the specifications [24,30].

In this study, all the individual networks chosen comply with the requirements of standard EN 326-2 in the 2000 [65] and 2014 [64] versions and would, therefore, be suitable for production control as an alternative to the standard method. The same occurs with the data obtained by bootstrapping the 20 neural networks: the correlation coefficient obtained is 0.98 and complies with standard EN 326-2 in the 2000 [65] version, and the p-value is 0.97, in compliance with the requirements of standard EN 326-2 in the 2014 [64] version, indicating that this ensemble is suitable for production control as an alternative to the standard method. However, no individual network provides the confidence interval of the modelling result. This means that bootstrapping must be applied, as indicated in Section 2.1, to obtain the confidence intervals.

It should also be noted that modelling with these neural networks allows considerable time saving in factory control, as the authors reported elsewhere [12,13], because conducting standard laboratory tests requires the mechanisation of the test pieces and conditioning before testing, whereas a neural network model based on production parameters would generate the test result in real time from values collected from production line sensors.

With regard to the degree of reliability of the networks obtained and their bootstrap ensemble compared to the reliability reported in other studies, both the determination coefficient (R²) and the correlation coefficient (R) obtained for the ANN (shown in Table 3) are lower than those reported in similar studies that modelled a mechanical property of a wood-based panel from production data. The smallest determination coefficient in the present study was 0.50, whereas Cook et al. [55] obtained determination coefficients greater than 0.8 for the mechanical properties of a particleboard panel calculated from production parameters by combining a radial-based ANN and a genetic algorithm. Demirkir et al. [68] reported determination coefficients greater than 0.97 when modelling the plywood bending strength from laboratory production data. Ozsahin [56] obtained determination coefficients of 0.88 and 0.92 when modelling the moisture absorption and swelling, respectively, of an OSB panel from the production data of the resin and additive type and content. Bardak, Tiryaki, Nemli, and Aydin [69] obtained a determination coefficient of 0.94 when modelling the bonding strength of wood joints from production data.

The authors of the present study also obtained higher determination coefficients in earlier studies with wood-based boards. García Fernández et al. [12] and Esteban et al. [20] reported determination coefficients of 0.75 to 0.85, respectively, when modelling particleboard mechanical properties using physical properties, and Esteban et al. [13] obtained determination coefficients of 0.92 and 0.94 when modelling the properties of a moisture-resistant particleboard after cyclic ageing. García Fernandez et al. [70] reported determination coefficients of 0.66 to 0.73 for the testing set when modelling the bending strength and modulus of the elasticity of structural plywood using other physico-mechanical properties. In a study on the influence of the physical properties on the internal bond of particleboard P2, they obtained a determination coefficient of 0.83 [60].

However, in all these investigations, most of the board production and the experiments to determine the values of the input and the output variables were carried out in the laboratory, whereas in the present study, all the input variables came from the production line and only the internal bond experiment was performed in controlled laboratory conditions. The higher correlation and determination coefficients obtained by these authors in earlier studies are probably due to the use of panels produced in the laboratory under very controlled production conditions that cannot always be replicated in daily factory production [4]. This is corroborated in the study by André et al. [71], who modelled the internal bond of fibreboard using data taken directly from the production line and reported correlation coefficients of 0.58 to 0.76 for the testing set, compared to the range of 0.71 to 0.81 in the present study. It, therefore, appears that the regression coefficients obtained in the present study are the result of using data and panels straight from the factory production line.

With regard to the combination of ANN by bootstrapping (Table 4), as Hosen et al. [36] and Hosen et al. [37] reported, the combined output of the 20 networks is greater than the outputs of the individual networks. In this case, the correlation coefficient improved from 0.95 in the best case, corresponding to the training set of the 12th network [7 2 1], to 0.98. These differences are more pronounced if the regressions are taken into account across all data, improving in this case from 0.87, corresponding to the regression of the total data of the first network [7 3 3], to 0.98. This shows how combining all the networks considerably improves the accuracy of the output.

For the quality of the intervals of confidence, the PICP (%) value obtained is 93% (Table 4), indicating that almost all the experimental data are included in the confidence interval. The PICP (%) value is similar to that of 95% reported by Mazloumi et al. [47] and by Khosravi et al. [72] in their study of the confidence intervals of energy market forecasts, where the PICP (%) were from 92.6 to 94.1%.

The value is also within the ranges obtained by Mazloumi et al. [47] and Khosravi et al. [25,32,33], who reported confidence intervals of 75 to 100% depending on the method used in the simulation. Hosen et al. [36] and Hosen et al. [37] obtained a very similar PICP of 82 to 94% and 80 to 95%, respectively, and Khosravi and Nahavandi [73] obtained a PICP higher than 95% from 50 simulations.

The result of this study also agrees with that of Shrivastava and Panigrahi [74] in their study of the confidence intervals of electricity market forecasts, where the PICP (%) was 50 to 100%, and with most of the results reported by Wan et al. [75], for a 95% coverage factor, they obtained a PICP (%) of 89.6 to 99.6% depending on the modelling method used.

Similarly, the PICP value agrees with the values obtained by Kasiviswanathan et al. [52], who referred to these values as POC of 92 to 97% when modelling rainfall runoff, although in critical situations, the PICP value was at times lower than 50%, and is slightly lower than the values reported by Kasiviswanathan and Sudheer [76], where the PICP values obtained for the bootstrap intervals ranged from 95.8 to 99.9%. It also agrees with the results of Hadjicharalambous et al. [26] in their study on neural network confidence intervals using various calculation methods (bootstrap, LUBE, and BLM), where the PICPs obtained were from 66 to 96% depending on the method used and, in the case of bootstrapping, from 66 to 83.23%.

Nourani et al. [77] also bootstrapped various neural networks to obtain the mean monthly precipitation and temperature at two weather stations (Tabriz and Ardabil, northwest Iran), using the bootstrap and LUBE methods, as Hadjicharalambous et al. [26] demonstrated. They obtained PICPs of 67 to 96% depending on the method used and the station and property modelled. These values are very similar to the PICP of 93% obtained in this study. In a later article on climatology, Nourani et al. [78] also studied the confidence intervals for models using artificial neural networks, generated using the methods of bootstrap, mean-variance estimation (MVE), delta, and LUBE. They reached the conclusion, previously indicated by Nerea Portillo et al. [38], Stoma et al. [39], and Khosravi et al. [33,34], that bootstrap intervals generate reliable confidence intervals. They also assessed the level of reliability of the intervals obtained through the PICP parameter, obtaining values higher than 95% for the bootstrap intervals, slightly higher than the 93% obtained in the present study.

Al-Dahidi et al. [79] similarly used the bootstrap technique to ensemble several artificial neural networks and obtain the prediction confidence intervals using the PICP parameter (%) to assess the prediction intervals. They modelled the prediction of the 24h-ahead solar photovoltaic production of a system of solar panels installed on the roof of the Faculty of Engineering at the Applied Science Private University (ASU) in Amman, Jordan. Their results were slightly lower than those obtained in the present study, as they reported a PICP of 84% for confidence intervals with a coverage factor of 80%, compared to the PICP of 93% for a coverage factor of 95% in the confidence intervals in the present study.

Seyedian et al. [80] studied the water discharge in an open channel, modelling the discharge using artificial neural networks, and obtained the intervals using the bootstrap from 17 initial data and a resampling set of 500 data. They also used the PICP parameter (%) for the final assessment of the confidence intervals, and obtained PICP values no higher than 92%, slightly lower than the 93% obtained in the present study.

More recently, in the oil field, Cheng et al. [81] studied the confidence intervals of the minimum miscibility pressure (MMP) of the CO₂-crude oil system, a critical parameter in assessing the efficiency of a CO₂ injection for enhanced oil recovery (EOR). They combined different modelling techniques using artificial intelligence with traditional models and assessed the result by the prediction interval coverage probability (PICP). The results obtained (PICP of 87 to 97%,) concur with the PICP of 93% obtained in the present study.

5. Conclusions

Modelling using a combination of neural networks is suitable for predicting the internal bond value of a standard particleboard from production parameters.

The reliability of the results is higher than the requirements of the standard for wood-based panels, indicating that this system is suitable for in-factory control.

Bootstrapping was used to calculate the confidence intervals of the output values of the value modelled, obtaining a 93% coverage factor in the results.

This study demonstrates the validity of using bootstrapping to calculate the confidence intervals of internal bond values obtained by simulation using a group of artificial neural networks.

The feasibility of using artificial neural networks for the in-factory control of the internal bond has been improved, given that the confidence intervals provide information about the degree of compliance with a specification when the results are close to the limits of the specification.

All of this would allow more accurate production control, based not only on having a modelling result that complies with the specification, but also on ensuring that the confidence interval of the result complies with the specification. This would bring production closer to product specification limits and help to ensure better the characterisation of the product and improve early failure detection on the production line.

Although the results obtained confirm the utility of bootstrapping for estimating confidence intervals in the quality control of wood-based boards using results modelled through artificial neural networks, this study has some limitations, as it addresses only the tensile strength perpendicular to the plane of the board, in a specific database. Therefore, given the growing integration of artificial intelligence techniques into quality control processes, future research must examine these methodologies of combining neural network modelling with traditional statistical techniques such as bootstraps in other physico-mechanical properties and different materials.