*2.3. Artificial Neural Networks*

An ANN represents a computational instrument that can "learn" to correctly map an input to an output via the adjustment of weights. The initial idea of the perceptron was to mimic the behaviour of a neuronal cell in the nervous system of the human brain [40]. Feed forward neural networks are multiple perceptrons composing one or more layers of neurons, where each neuron computes an output based on inputs from the previous layer and an inherent non-linear activation function. The signal is processed in an unidirectional forward direction from input to output throughout the network, where the input signal is progressively transformed into an output signal, see Figure 4. ANNs can be trained to approximate any non-linear relationship [41]. Training of such networks is achieved through back propagating error minimization via gradient descent. The error resulting from the difference between current network output and true/desired output (which is known in a supervised learning task) is minimized by adapting the behaviour of individual neurons through adjusting the weights of the connecting edges between those neurons. The learning rate defines the step size per weight update during gradient descent. For the implementation of an adaptive learning rate, different learning rate optimizers are available, such as Adam [42], Momentum [43] or Adagrad [44], among others. Ultimately, the network represents a mapping rule that is based on provided training examples and is only valid for the space contained in those samples; thus, these networks are not suitable for extrapolating predictions outside the training sample domain. A brief description of a feed forward neural network with back propagating error minimization is provided in the following.

Overall, achieving sufficient training and validation of an ANN depends on the amount of available data, network complexity and the nonlinear nature of the particular relationships to be approximated. To obtain a good ability of the ANN to generalize well, the prediction error on training and validation data sets need to be both low and similar [45], as it indicates that neither underfitting nor overfitting has occurred during training. To prevent overfitting on the training data, learning can be terminated based on the "early stopping" criterion, which is fulfilled as soon as the prediction performance on the validation data set (outside the training data set) is no longer improved during training, even though the error on the training set is still decreasing.

**Figure 4.** Schematic of a multi-layered neural network with input layer, *k* hidden layer and output layer, including weight vectors *W* of edge connections between neurons of adjacent layers for correlating *n* number of inputs [*x*<sup>1</sup> , *x*2, ..., *xn*] to *m* number of outputs [*y*<sup>1</sup> , *y*2, ..., *ym*].

#### **3. Methodology**

First, patterns are generated with pressure pulses and residual stresses from both semi-analytical and FE models. The resulting pairs of semi-analytically and numerically determined residual stress profiles compose the training data set for the corrective task of the ANN. Second, the ANN is trained, validated and tested. Third, the ANN is utilized for correcting semi-analytical residual stress profiles generated by an expanded pulse parameter range that was not contained in the previously utilized training, validation and test data sets. This methodology is described in detail in the following.

As illustrated in Figure 5, the corrected predictions for LSP-induced residual stresses contain the estimates from the physics-based semi-analytical model and a corrective term from the corrective ANN that accounts for the deviation between semi-analytical stresses and numerical stresses to generate the desired high-fidelity solution.

**Figure 5.** Schematic of hybrid model implementation for prediction of laser shock peening (LSP)-induced residual stresses: (**a**) Residual stresses predicted by the semi-analytical model exhibiting relatively high prediction errors compared to the high fidelity FE solution which is compensated by (**b**) a correction factor "learned" by an artificial neural network (ANN), leading to (**c**) the validated high-fidelity prediction with low errors, i.e., the hybrid model solution.
