Statistical Metamodel of Liner Acoustic Impedance Based on Neural Network and Probabilistic Learning for Small Datasets

Abstract

The main novelty of this paper consists of presenting a statistical artificial neural network (ANN)-based model for a robust prediction of the frequency-dependent aeroacoustic liner impedance using an aeroacoustic computational model (ACM) dataset of small size. The model, focusing on percentage of open area (POA) and sound pressure level (SPL) at a zero Mach number, takes into account uncertainties using a probabilistic formulation. The main difficulty in training an ANN-based model is the small size of the ACM dataset. The probabilistic learning carried out using the probabilistic learning on manifolds (PLoM) algorithm addresses this difficulty as it allows constructing a very large training dataset from learning the probabilistic model from a small dataset. A prior conditional probability model is presented for the PCA-based statistical reduced representation of the frequency-sampled vector of the log-resistance and reactance. It induces some statistical constraints that are not straightforwardly taken into account when training such an ANN-based model by classical optimizations methods under constraints. A second novelty of this paper consists of presenting an alternate solution that involves using conditional statistics estimated with learned realizations from PLoM. A numerical example is presented.

1. Introduction

We are interested in aircraft noise in the context of green aviation. This includes both external and internal noise relative to the aircraft for certification purposes. In modern turbofan engines, especially those with an ultra-high bypass ratio (UHBR), fan noise is a major contributor to overall noise. Fan noise comprises broadband and tonal noise. To attenuate the tonal noise component, particularly at a blade passing frequency (BPF), targeted acoustic liners are used, while modifications to the liners’ geometry and intrinsic properties help dissipate the broadband noise component. For effectiveness, it is crucial to study liners under various operating conditions, including different flight scenarios. The design of liners is critically important and has been extensively studied (see [1,2,3,4,5,6,7]). Liner acoustic impedance can be calculated by using an aeroacoustic computational model (ACM), which is a computationally expensive and prohibitive high-fidelity computational model for optimizing aeroacoustic performances of liners using a numerical optimization solver. Such ACMs have been developed to predict liner performance, as detailed in [8,9,10,11,12,13]. An uncertain ACM of the liner system, which allows for quantifying uncertainties in aeroacoustic models of liner performance, was also developed in [6].

The objective of this paper is to construct a statistical metamodel for which the outputs are the frequency-sampled impedance of the liner and the inputs are the control parameters that are the design parameters. The metamodel should have a low computational cost to enable its usage in optimization of liners’ aeroacoustic performance via a numerical optimization solver. In addition, the gradients of the metamodel with respect to its inputs (the control parameters) should also have a low computational cost.

In this paper, we construct the metamodel using a dataset, referred to as the ACM dataset, which includes samples of control parameters and the corresponding acoustic impedance, numerically simulated with the ACM. Consequently, the dataset is small due to the prohibitive computational cost of the ACM, which prevents the construction of a large dataset.

As only the components of the control parameter are inputs to the metamodel, all other ACM parameters are unobserved (thus uncontrolled) and should be treated as random latent variables. Therefore, the liner acoustic impedance at any frequency should also be modeled as a random variable. A first novelty contribution of this paper is the methodology for constructing such a statistical metamodel that is driven by the physics contained in the small ACM dataset.

In this paper, the statistical metamodel is defined by the conditional probability distribution of the frequency-sampled vector of the random acoustic impedance at sampled frequencies, given the control parameter. The MaxEnt principle, applied with the available information, is used to construct an informative prior model of this conditional probability distribution. It should be noted that the hyperparameters of the probabilistic model depend on the values of the control parameter and are modeled using fully connected feedforward neural networks, yielding a statistical ANN-based metamodel. Such a statistical ANN-based metamodel is fitted on an ad hoc training dataset using the maximum likelihood principle.

Principal component analysis (PCA) is conducted on the outputs (frequency-sampled acoustic impedance) of the metamodel. This is not only carried out for a potential statistical reduction but also because decorrelation and centering of outputs enhance numerical conditioning, thereby facilitating the optimization process for fitting such a statistical ANN-based model to the training dataset. However, such a statistical decorrelation and centering of the outputs introduce constraints on the hyperparameters of the statistical metamodel and, consequently, on the parameters of the statistical ANN-based metamodel. Therefore, given that these hyperparameters are modeled by fully connected feedforward networks, some deterministic constraints must be considered, yielding the development of constrained training algorithm for such networks. Such a constrained training algorithm is challenging and complex when dealing with mini-batches, and might rely on techniques such as Lagrange multipliers, penalization approaches, correction formulations, etc. A second novelty of this paper is the presentation of an unconstrained formulation for a statistical ANN-based metamodel that takes into account the statistical constraints arising from the PCA-based reduction. This is achieved by generating an ad hoc training dataset using probabilistic learning on manifolds [14,15,16,17] and using the learned dataset for estimating conditional statistics in a nonparametric framework, based on the Gaussian kernel density estimation (GKDE) method. Finally, the statistical metamodel can be used to generate additional realizations of the frequency-sampled acoustic impedance vector, thereby mitigating missing data in the set of control parameters. Compared to a previous work [18], this ANN-based surrogate model is actually a complement. First, it does not require the training dataset for offline use, significantly reducing memory requirements. Second, it allows for more efficient computation of gradients with respect to input parameters, which is crucial for optimization problems.

Note that the objective of this paper is not to improve the ANN algorithms but is to present a novel methodological application of ANNs to probabilistic ACM datasets for aeroacoustic liner impedance simulations. The novelty lies in how ANNs are used to model the hyperparameters of the probabilistic model (Section 4.1 and Section 4.2) and in how to train this model using a learned GKDE-based estimates’ dataset by PLoM (Section 4.3). This approach allows us to effectively deal with a small and probabilistic dataset, which is a significant challenge in this field and has not been widely addressed in the literature yet.

This paper is organized as follows: (1) Section 2 briefly defines the control parameters and the ACM for calculating the frequency-sampled vector of acoustic impedance, which is to be used for constructing the ACM dataset (small size). (2) Section 3 is dedicated to the parametric probabilistic modeling of the conditional probability distribution of the frequency-sampled vector of acoustic impedance, given the control parameter. (3) Section 4 focuses on the statistical ANN-based metamodel, given the control parameter. (4) Section 5 presents a numerical example, along with a discussion of the results.

2. Control Parameters and ACM Dataset

In this paper, the considered system is an acoustic liner consisting of a perforated plate, a honeycomb structure, and a rigid backing plate, as depicted in Figure 1. The Mach number is assumed to be equal to zero, which is relevant for the ground configuration. Incorporating non-zero Mach numbers requires additional analyses beyond the scope of this paper. A reduced domain, as described in [8], is used for the computational model (see Figure 2). In this paper, and for sake of simplicity, the liner system is parameterized by $n_{\mathbf{w}}=2$ parameters (the control parameters) that are the percentage of open area (POA) and the sound pressure level (SPL), represented by $\mathbf{w}=(w_1,w_2)$, where $w_1$ corresponds to the POA and $w_2$ to the SPL.

For such a reduced domain, the frequency-dependent acoustic impedance is denoted as $\omega \mapsto {\mathbf{z}}^{\mathrm{acm}}(\omega ;\mathbf{w})$, where $\omega$ is the frequency (rad/s). Specifically, ${\mathbf{z}}^{\mathrm{acm}}(\omega ;\mathbf{w})=r^{\mathrm{acm}}(\omega ;\mathbf{w})+\iota v^{\mathrm{acm}}(\omega ;\mathbf{w})$, in which $r^{\mathrm{acm}}(\omega ;\mathbf{w})$ represents the resistance that is positive, $v^{\mathrm{acm}}(\omega ;\mathbf{w})$ the reactance that is real, and $\iota =\sqrt{-1}$. Control parameter $\mathbf{w}$ belongs to an admissible set. The computational domain is centered around the resonator, within which the Navier–Stokes equations are solved. The computational model consists of 278,514 degrees of freedom. For given control parameter $\mathbf{w}$, and for each sampled frequency ${\omega }_k=(k-1)\Delta \omega$ with $k=1,\dots ,n_{\mathbf{w}}$ where $n_{\omega }=7$, the ACM computes the resistance $r^{\mathrm{acm}}({\omega }_k;\mathbf{w})$ and the reactance $v^{\mathrm{acm}}({\omega }_k;\mathbf{w})$. The Navier–Stokes equations are solved using the numerical method presented in [8] for $\mathbf{w}=(w_1,w_2)\in \left[0.03,0.1\right]\times \left[130,145\right]$. As discussed in [18], ACM simulations are performed for $n_d=48$ values ${\mathbf{w}}^1,\dots ,{\mathbf{w}}^{n_d}$ of $\mathbf{w}$, which constitute the set ${\mathcal{D}}_{\mathbf{w}}$. These points in ${\mathcal{D}}_{\mathbf{w}}$ are considered as realizations of a random vector $\mathbf{W}$ whose probability density function, $p_{\mathbf{W}}$, is unknown.

The ACM dataset ${\mathcal{D}}_{\mathrm{acm}}$ is then defined as the set of points $({\mathbf{w}}^j,{\mathbf{r}}^{\mathrm{acm},j},{\mathbf{v}}^{\mathrm{acm},j})$ in ${\mathbb{R}}^{n_{\mathbf{w}}\times n_{\omega }\times n_{\omega }}$ for $j=1,\dots ,n_d$, in which ${\mathbf{r}}^{\mathrm{acm},j}=(r^{\mathrm{acm}}({\omega }_1;{\mathbf{w}}^j),\dots ,r^{\mathrm{acm}}({\omega }_{n_{\omega }};{\mathbf{w}}^j))$ and ${\mathbf{v}}^{\mathrm{acm},j}=\left(v^{\mathrm{acm}}\right({\omega }_1;$${\mathbf{w}}^j),\dots ,v^{\mathrm{acm}}({\omega }_{n_{\omega }};{\mathbf{w}}^j))$. For each $j=1,\dots ,n_d$, we introduce the $2n_{\omega }$-dimensional vector of frequency-sampled impedance ${\mathbf{q}}^{\mathrm{acm},j}=(log{\mathbf{r}}^{\mathrm{acm},j},{\mathbf{v}}^{\mathrm{acm},j})$ in which $log{\mathbf{r}}^{\mathrm{acm},j}=(log{r}^{\mathrm{acm}}({\omega }_1;{\mathbf{w}}^j),\dots ,log{r}^{\mathrm{acm}}({\omega }_{n_{\omega }};{\mathbf{w}}^j))$.

The model uncertainties are due to random latent parameters that, consequently, cannot be defined as control parameters. In order to take into account these model uncertainties, the random vectors $\mathbf{Q},\mathbf{R}$, and $\mathbf{V}$ are introduced, whose conditional probability density functions, given $\mathbf{W}={\mathbf{w}}^j$, are denoted by $\mathbf{q}\mapsto p_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{q}\right|{\mathbf{w}}^j)$, $p_{\mathbf{R}|\mathbf{W}}\left(\mathbf{r}\right|{\mathbf{w}}^j)$, and $p_{\mathbf{V}|\mathbf{W}}\left(\mathbf{v}\right|{\mathbf{w}}^j)$. These conditional probability density functions are constructed as explained in [18]. For instance, the conditional mean value of $\mathbf{Q}$ given $\mathbf{W}={\mathbf{w}}^j$ is chosen as ${\mathbf{q}}^{\mathrm{acm},j}$ for $j=1,\dots ,n_d$; the conditional dispersion coefficient given $\mathbf{W}={\mathbf{w}}^j$ is identified using experimental data. We then generate $m_d=15$ statistically independent realizations ${\mathbf{q}}^{j,1},\dots ,{\mathbf{q}}^{j,m_d}$ from $p_{\mathbf{Q}|\mathbf{W}}$, given $\mathbf{W}={\mathbf{w}}^j$. Hence, training dataset ${\mathcal{D}}_{\mathrm{acm}}^{\ast }$ is made up of a total of $n_{\mathrm{acm}}=n_d\times m_d$ realizations $({\mathbf{w}}^{j,k},{\mathbf{q}}^{j,k})$ with $j=1,\dots ,n_d$ and $k=1,\dots ,m_d$, in which ${\mathbf{w}}^{j,k}$ is a rewriting of ${\mathbf{w}}^j$ that is independent of k (we introduce a repetition). For the sake of simplicity, all the realizations are rewritten as $({\mathbf{w}}^j,{\mathbf{q}}^j)$ with $j=1,\dots ,n_{\mathrm{acm}}$.

In [18], the PLoM (probabilistic learning on manifolds) is carried out to learn the joint probability density function $p_{\mathbf{Q},\mathbf{W}}$ of random vectors $\mathbf{Q}$ and $\mathbf{W}$ using ${\mathcal{D}}_{\mathrm{acm}}^{\ast }$ as a training dataset. PLoM also allows $n_{\mathrm{ar}}$ additional statistically independent realizations $\{({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }},{\mathbf{q}}_{\mathrm{ar}}^{\mathsf{\ell }}),\mathsf{\ell }=1,\dots ,n_{\mathrm{ar}}\}$ to be generated, which constitute the learned dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$.

3. Prior Probabilistic Model of the Frequency-Sampled Impedance Vector

In [18], the conditional probability density function $p_{\mathbf{Q}|\mathbf{W}}$ is estimated with Gaussian kernel density estimation (GKDE) using additional realizations (generated by PLoM) of $\mathbf{Q}$ and $\mathbf{W}$. In the context of the construction of a statistical metamodel based on a neural network, we need to introduce an algebraic representation of the conditional probability distribution of $\mathbf{Q}$ given $\mathbf{W}$, depending on hyperparameters. In this paper, we have chosen a Gaussian model for which the hyperparameters are conditional mean value and the conditional covariance matrix of $\mathbf{Q}$, given $\mathbf{W}$. The neural network will be used for predicting these conditional hyperparameters. In this section, we then present the construction of the algebraic priorprobabilistic model of $p_{\mathbf{Q}|\mathbf{W}}$. Nevertheless, since $\mathbf{Q}$ is in a high dimension, we will introduce a statistical reduction $\mathbf{H}$ of $\mathbf{Q}$ using PCA. Within Section 3.2, the conditional hyperparameters associated with such a prior probabilistic model are represented as functions of $\mathbf{w}$. The modeling of these functions is carried out using fully connected feedforward networks that are trained to map control parameter $\mathbf{w}$ onto a corresponding set of hyperparameters of the priorprobabilistic model.

3.1. PCA-Based Statistical Reduction $\mathit{H}$ of $\mathit{Q}$

PCA is used to construct the statistical reduction of $\mathbf{Q}$, yielding a normalized random vector $\mathbf{H}$ (centered with identity covariance matrix). Random vector $\mathbf{H}$ is therefore written as $\mathbf{H}={\left[\lambda \right]}^{-1/2}{\left[\varphi \right]}^T\left(\mathbf{Q}-\underline{\mathbf{q}}\right)$, in which $\underline{\mathbf{q}}=(1/n_{\mathrm{ar}}){\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}{\mathbf{q}}_{\mathrm{ar}}^{\mathsf{\ell }}$ is the empirical mean value of random vector $\mathbf{Q}$, $\left[\lambda \right]$ is a $(m\times m)$ diagonal matrix, and $\left[\varphi \right]$ is a $(n_q\times m)$ matrix whose columns are orthonormal vectors, and are such that $\left[C_{\mathbf{Q}}\right]\left[\varphi \right]=\left[\varphi \right]\left[\lambda \right]$. The estimate of the $(n_{\mathbf{q}}\times n_{\mathbf{q}})$ covariance matrix of $\mathbf{Q}$ is $\left[C_{\mathbf{Q}}\right]={(n_{\mathrm{ar}}-1)}^{-1}{\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}({\mathbf{q}}_{\mathrm{ar}}^{\mathsf{\ell }}-\underline{\mathbf{q}}){({\mathbf{q}}_{\mathrm{ar}}^{\mathsf{\ell }}-\underline{\mathbf{q}})}^T$. The diagonal entries of $\left[\lambda \right]$ are the m-largest eigenvalues of $\left[C_{\mathbf{Q}}\right]$. By construction, the ${\mathbb{R}}^m$-valued random variable $\mathbf{H}$ is such that

$$E\left\{\mathbf{H}\right\}=0_m,E\{\mathbf{H}\otimes \mathbf{H}\}=\left[{\mathrm{I}}_m\right].$$

(1)

3.2. Prior Conditional Probabilistic Density Function of $\mathit{H}$ Given $\mathit{W}$

Let $\mathsf{\eta }\mapsto p_{\mathbf{H}|\mathbf{W}}\left(\mathsf{\eta }\right|\mathbf{w})$ be the conditional probability density function of $\mathbf{H}$, given $\mathbf{W}$. The priorconditional probability density function $p_{\mathbf{H}|\mathbf{W}}$ is constructed using the MaxEnt principle (see, for instance, [19]) with the following available information: (1) The support of $\mathsf{\eta }\mapsto p_{\mathbf{H}|\mathbf{W}}\left(\mathsf{\eta }\right|\mathbf{w})$ is ${\mathbb{R}}^m$. (2) The conditional mean value and the conditional covariance matrix of $\mathbf{H}$ given $\mathbf{W}=\mathbf{w}$ are the vector ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ and the matrix $\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$, which are estimated using the training dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$ for each given value of $\mathbf{w}$. Therefore, $\mathbf{H}$ given $\mathbf{W}=\mathbf{w}$ is a multivariate Gaussian random variable with mean value ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ and covariance matrix $\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$.

For any given $\mathbf{w}$ in its admissible set, the estimates of hyperparameters ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ and $\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$, constructed using GKDE from nonparametric statistics and dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$, are written as

$${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)={\displaystyle \frac{{\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}{\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}exp\left(-\frac{1}{2s^2}{(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}{{\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}exp\left(-\frac{1}{2s^2}{(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}},$$

(2)

$$\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]={\displaystyle \frac{{\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}{\tilde{\mathsf{\eta }}}_{\mathrm{ar}}^{\mathsf{\ell }}\left(\mathbf{w}\right){\left({\tilde{\mathsf{\eta }}}_{\mathrm{ar}}^{\mathsf{\ell }}\left(\mathbf{w}\right)\right)}^{T}exp\left(-\frac{1}{2s^2}{(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}{{\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}exp\left(-\frac{1}{2s^2}{(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}(\mathbf{w}-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}},$$

(3)

where (1) the $(n_{\mathbf{w}}\times n_{\mathbf{w}})$ matrix $\left[C_{\mathbf{W}}\right]={(n_{\mathrm{ar}}-1)}^{-1}{\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }}-\underline{\mathbf{w}}){({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }}-\underline{\mathbf{w}})}^T$ is the estimate of the covariance matrix of $\mathbf{W}$ in which $\underline{\mathbf{w}}=(1/n_{\mathrm{ar}}){\sum }_{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }}$ is the estimate of the mean value of $\mathbf{W}$; (2) for all $\mathsf{\ell }=1,\dots ,n_{\mathrm{ar}}$, we have ${\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}={\left[\lambda \right]}^{-1/2}{\left[\varphi \right]}^T({\mathbf{q}}_{\mathrm{ar}}^{\mathsf{\ell }}-\underline{\mathbf{q}})$ and ${\tilde{\mathsf{\eta }}}_{\mathrm{ar}}^{\mathsf{\ell }}\left(\mathbf{w}\right)={\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}-{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$; (3) s is the Silverman bandwidth given by

$$s={\left\{{\displaystyle \frac{4}{n_{\mathrm{ar}}(2+n)}}\right\}}^{1/(n+4)},n=m+n_{\mathbf{w}}$$

(4)

Due to Equation (1), ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ and $\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ have to satisfy the following equations:

$$E\left\{{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{W}\right)\right\}=0_m,$$

(5)

$$E\left\{\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{W}\right)\right]+{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{W}\right){\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}{\left(\mathbf{W}\right)}^T\right\}=\left[I_m\right].$$

(6)

With the proposed methodology, these two equations will automatically be satisfied.

3.3. Statistically Independent Realizations of $\mathit{R}$ and $\mathit{V}$ Given $\mathit{W}$

For given $\mathbf{w}$, let $\mathbf{R}\left(\mathbf{w}\right)$ and $\mathbf{V}\left(\mathbf{w}\right)$ be the ${\mathbb{R}}^{n_{\omega }}$-valued random variables defined in Section 2. For any given $\mathbf{w}$ in its admissible set, N statistically independent realizations ${\mathsf{\eta }}^1\left(\mathbf{w}\right),\dots ,{\mathsf{\eta }}^N\left(\mathbf{w}\right)$ of $\mathbf{H}$ given $\mathbf{W}=\mathbf{w}$ are generated using the multivariate Gaussian random distribution whose mean value is ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ and covariance matrix is $\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$. We then deduce N statistically independent realizations ${\mathbf{q}}^1\left(\mathbf{w}\right),\dots ,{\mathbf{q}}^N\left(\mathbf{w}\right)$ of random vector $\mathbf{Q}$ given $\mathbf{W}=\mathbf{w}$ such that, for $j=1,\dots ,N$, ${\mathbf{q}}^j\left(\mathbf{w}\right)=\underline{\mathbf{q}}+\left[\mathsf{\Phi }\right]{\left[\lambda \right]}^{1/2}{\mathsf{\eta }}^j\left(\mathbf{w}\right)$. For $j=1,\dots ,N$, the block decomposition of vector ${\mathbf{q}}^j\left(\mathbf{w}\right)$ is written as $({\mathbf{q}}_{\mathbf{R}}^j\left(\mathbf{w}\right),{\mathbf{v}}^j\left(\mathbf{w}\right))$ with ${\mathbf{q}}_{\mathbf{R}}^j$ and ${\mathbf{v}}^j\left(\mathbf{w}\right)$ being two $n_{\omega }$ dimensional vectors. Obviously, ${\mathbf{v}}^1\left(\mathbf{w}\right),\dots ,{\mathbf{v}}^N\left(\mathbf{w}\right)$ are statistically independent realizations of $\mathbf{V}\left(\mathbf{w}\right)$, and the statistically independent realizations ${\mathbf{r}}^1\left(\mathbf{w}\right),\dots ,{\mathbf{r}}^N\left(\mathbf{w}\right)$ of $\mathbf{R}\left(\mathbf{w}\right)$ are such that ${\mathbf{q}}_{\mathbf{R}}^j\left(\mathbf{w}\right)=log\left({\mathbf{r}}^j\left(\mathbf{w}\right)\right)$. Note that, for a given $\mathbf{w}$, conditional mean vectors ${\mathsf{\mu }}_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)$ and ${\mathsf{\mu }}_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)$, as well as the conditional covariance matrices $\left[C_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ and $\left[C_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)\right]$, are estimated using the statistically independent realizations ${\mathbf{r}}^j\left(\mathbf{w}\right),\dots ,{\mathbf{r}}^N\left(\mathbf{w}\right)$ and ${\mathbf{v}}^j\left(\mathbf{w}\right),\dots ,{\mathbf{v}}^N\left(\mathbf{w}\right)$. Note that vector ${\mathsf{\mu }}_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)$ and matrix $\left[C_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ can also be written as

$${\mathsf{\mu }}_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)=exp\left({\mathsf{\mu }}_{{\mathbf{Q}}_{\mathbf{R}}|\mathbf{W}}\left(\mathbf{w}\right)+{\displaystyle \frac{1}{2}}diag\left[C_{{\mathbf{Q}}_{\mathbf{R}}|\mathbf{W}}\left(\mathbf{w}\right)\right]\right),$$

(7)

$$\left[C_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)\right]=exp\left(\left[C_{{\mathbf{Q}}_{\mathbf{R}}|\mathbf{W}}\left(\mathbf{w}\right)\right]-1\right)\odot \left({\mathsf{\mu }}_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right){\mathsf{\mu }}_{\mathbf{R}|\mathbf{W}}{\left(\mathbf{w}\right)}^T\right),$$

(8)

where $\left[A\right]\odot \left[B\right]$ stands for Hadamard product of matrices $\left[A\right]$ and $\left[B\right]$; $exp\left(\right[A\left]\right)$ is the element-wise exponential, and $diag\left[A\right]$ is the vector made up of the diagonal entries of a given matrix $\left[A\right]$. The block decomposition of ${\mathsf{\mu }}_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{w}\right)=\underline{\mathbf{q}}+\left[\mathsf{\Phi }\right]{\left[\lambda \right]}^{1/2}{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ into $n_{\omega }$ dimensional vectors ${\mathsf{\mu }}_{{\mathbf{Q}}_{\mathbf{R}}|\mathbf{W}}\left(\mathbf{w}\right)$ and ${\mathsf{\mu }}_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)$ is written as

$${\mathsf{\mu }}_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{w}\right)=({\mathsf{\mu }}_{{\mathbf{Q}}_{\mathbf{R}}|\mathbf{W}}\left(\mathbf{w}\right),{\mathsf{\mu }}_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)).$$

(9)

The block decomposition of matrix $\left[C_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{w}\right)\right]=\left[\mathsf{\Phi }\right]{\left[\lambda \right]}^{1/2}\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]{\left[\lambda \right]}^{1/2}{\left[\mathsf{\Phi }\right]}^T$ into $(n_{\omega }\times n_{\omega })$ matrices $\left[C_{{\mathbf{Q}}_{\mathbf{R}}|\mathbf{W}}\left(\mathbf{w}\right)\right]$, $\left[C\right(\mathbf{w}\left)\right]$, and $\left[C_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ is written as

$$\left[C_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{w}\right)\right]=\left(\begin{array}{ccccc}& \left[C_{{\mathbf{Q}}_{\mathbf{R}}|\mathbf{W}}\left(\mathbf{w}\right)\right]& & \left[C\right(\mathbf{w}\left)\right]& \\ & & & & \\ & \left[C\right(\mathbf{w}\left)\right]& & \left[C_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)\right]& \end{array}\right).$$

(10)

Concerning the random frequency-sampled vector $\mathbf{V}\left(\mathbf{w}\right)$, vector ${\mathsf{\mu }}_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)$ and matrix $\left[C_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ are directly obtained from the block decomposition of vector ${\mathsf{\mu }}_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{w}\right)$ and matrix $\left[C_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ given by Equations (9) and (10).

4. Statistical ANN-Based Metamodel

4.1. Fully Connected Feedforward Neural Network

Deterministic mappings $\mathbf{w}\mapsto {\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ and $\mathbf{w}\mapsto \left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ may have complex behavior, not only because their supports are multidimensional, but also because the underlying physical process is complex. In such a case, a fully connected feedforward neural network is well adapted to represent such deterministic mappings. We then consider a fully connected feedforward neural network ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}{\mathsf{\theta }}_1\right)$ with parameter ${\mathsf{\theta }}_1$, which is constructed in order to model deterministic mapping $\mathbf{w}\mapsto {\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$. However, a representation of $\mathbf{w}\mapsto \left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ by a fully connected feedforward network is not straightforward because, for each given $\mathbf{w}$, the output has to be a positive-definite matrix. To circumvent this apparent difficulty, the matrix logarithm of the $(m\times m)$ symmetric covariance matrix $\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ is calculated for each given $\mathbf{w}$, which yields a $(m\times m)$ symmetric matrix $[log{C}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)]$. If all the $m(m+1)/2$ entries of the upper triangular block of matrix $[log{C}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)]$ are collected into the $m(m+1)/2$ dimensional vector ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$, then a fully connected feedforward network ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}{\mathsf{\theta }}_2\right)$ with parameter ${\mathsf{\theta }}_2$ is constructed for modeling the deterministic mapping $\mathbf{w}\mapsto {\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$. Then, for a given $\mathbf{w}$, the output of ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)$ with parameter ${\mathsf{\theta }}_2$ is used in order to assemble matrix $[log{C}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)]$. Then, matrix $\left[C_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)\right]$ is calculated as the matrix exponential of $[log{C}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)]$. We then define the statistical ANN-based metamodel as the probabilistic model in Section 3, where ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1)$ and $\left[C_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)\right]$ are used for modeling ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)$ and $\left[C_{\mathbf{H}|\mathbf{W}}\left(\mathbf{w}\right)\right]$. Consequently, the statistical ANN-based model is defined as the probability density function $\mathsf{\eta }\mapsto p_{\mathbf{H}|\mathbf{W}}\left(\mathsf{\eta }\right|\mathbf{w};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$ that is a multivariate Gaussian probability density function with mean value ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1)$ and covariance matrix $\left[C_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)\right]$.

This ANN-based approach is actually a complement to the model presented in [18]. The previous model relies on conditional statistics based on the PLoM generation of the learned dataset requiring the training dataset. The proposed ANN-based model, once trained, can make predictions without referencing the learned dataset, at the price of introducing a probabilistic simplification concerning the probability distributions. This feature significantly reduces the computational resources required for deployment. In addition, the ANN-based model can be viewed as another type of representation of the surrogate model in [18].

4.2. Statistical ANN-Based Metamodel for Regression with the Learned Dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$

As it is usually the case for most regression problems, parameters ${\mathsf{\theta }}_1$ and ${\mathsf{\theta }}_2$ are adjusted by fitting the statistical ANN-based metamodel to a suitable training dataset. In this paper, such fitting is carried out in minimizing with respect to ${\mathsf{\theta }}_1$ and ${\mathsf{\theta }}_2$ regarding the negative-log-likelihood $\mathcal{L}({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$ of the statistical ANN-based model. Obviously, ${\mathcal{D}}_{\mathrm{acm}}^{\ast }$ and ${\mathcal{D}}_{\mathrm{acm}}$ are not suitable as training datasets for such a fitting process since their sizes are small (less than one hundred elements). It is well known that, in the framework of statistical ANN-based modeling (as well for deterministic ANN-based modeling), small training datasets yields overfitting models that are unable to predict their targets well enough for unseen values of their inputs. On the other hand, since the size $n_{\mathrm{ar}}$ of learned dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$ is as large as needed, such dataset is completely suitable as a training dataset. Using learned dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$, the negative-log-likelihood to be minimized is written as

$$\begin{array}{ccc}\hfill \mathcal{L}({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)& =& -\sum _{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}log{p}_{\mathbf{H},\mathbf{W}}({\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }},{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2),\hfill \end{array}$$

(11)

$$\begin{array}{ccc}\hfill & =& -\sum _{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}log\left(p_{\mathbf{H}|\mathbf{W}}\left({\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}\right|{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2)p_{\mathbf{W}}\left({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }}\right)\right),\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill & =& -\sum _{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}log{p}_{\mathbf{H}|\mathbf{W}}\left({\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}\right|{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2)-\sum _{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}log{p}_{\mathbf{W}}\left({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }}\right).\hfill \end{array}$$

(13)

Minimizing $\mathcal{L}({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$ with respect to ${\mathsf{\theta }}_1$ and ${\mathsf{\theta }}_2$ is equivalent to minimizing the cost function $J({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$, which is defined by

$$\begin{array}{cc}J({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)\hfill & ={\displaystyle \frac{1}{2}}\sum _{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}log\left(det\left[C_{\mathbf{H}|\mathbf{W}}({\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }};{\mathsf{\theta }}_2)\right]\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\sum _{\mathsf{\ell }=1}^{n_{\mathrm{ar}}}{\left({\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}-{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }};{\mathsf{\theta }}_1)\right)}^T{\left[C_{\mathbf{H}|\mathbf{W}}({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }};{\mathsf{\theta }}_2)\right]}^{-1}\left({\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}-{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}({\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }};{\mathsf{\theta }}_1)\right).\hfill \end{array}$$

(14)

Note that such minimization should be performed under the constraints defined by Equations (5) and (6). Classically, such constraints would be taken into account by introducing Lagrange multipliers [20,21,22], augmented Lagrangian methods [21,22], penalty methods [20,21,22], barrier methods [22], projected gradient methods [23], or Sequential Quadratic Programming [22]. It is not straightforward to implement such methods in the framework of fully connected feedforward neural network training when mini-batches are required due to constraints regarding RAM availability due to CPU or GPU limitations. In this paper, and as explained in Section 4.3, we take advantage of the PLoM in order to fit the probabilistic model by adjusting ${\mathsf{\theta }}_1$ and ${\mathsf{\theta }}_2$, such that constraints defined by Equations (5) and (6) are automatically satisfied.

4.3. Statistical ANN-Based Metamodel for Regression with a Learned GKDE-Based Estimates’ Dataset

It should be noted that minimizing $J({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$ with respect to ${\mathsf{\theta }}_1$ and ${\mathsf{\theta }}_2$ under constraints defined by Equations (5) and (6) is equivalent to constructing the likelihood-based statistical estimators of the conditional mean and conditional covariance matrix of $\mathbf{H}$, given $\mathbf{W}=\mathbf{w}$. Such statistical estimators are different from the GKDE-based statistical estimators defined by Equations (2) and (3). Therefore, in the context of constructing ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1)$ and ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)$, an alternative strategy to the typical approach of minimizing the negative-log-likelihood $\mathcal{L}({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$ in regression problems, as presented in Section 4.2, is proposed. This alternative entails generating a GKDE-based estimates’ dataset ${\mathcal{D}}_{\mathbf{H}|\mathbf{W}}^{\ast }$ made up of pre-calculated estimates of a conditional mean and conditional covariance of $\mathbf{H}$, given $\mathbf{W}$, using GKDE-based estimators as defined by Equations (2) and (3). Then, parameters ${\mathsf{\theta }}_1$ and ${\mathsf{\theta }}_2$ are adjusted such that the statistical ANN-based metamodel defined by ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1)$ and ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)$ fits dataset ${\mathcal{D}}_{\mathbf{H}|\mathbf{W}}^{\ast }$. Such a strategy requires a very large dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$ in order to construct a large enough dataset ${\mathcal{D}}_{\mathbf{H}|\mathbf{W}}^{\ast }$ using GKDE-based estimators. Such an adapted very large dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$ can easily be constructed by PLoM. Therefore, dataset ${\mathcal{D}}_{\mathbf{H}|\mathbf{W}}^{\ast }$ consists of $\nu <n_{\mathrm{ar}}$ elements $({\mathsf{\mu }}^1,{\mathsf{\zeta }}^1),\dots ,({\mathsf{\mu }}^{\nu },{\mathsf{\zeta }}^{\nu })$ defined as follows. For all $j=1,\dots ,\nu$,

$${\mathsf{\mu }}^j={\displaystyle \frac{{\sum }_{\mathsf{\ell }=\nu +1}^{n_{\mathrm{ar}}}{\mathsf{\eta }}_{\mathrm{ar}}^{\mathsf{\ell }}exp\left(-\frac{1}{2s^2}{({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}{{\sum }_{\mathsf{\ell }=\nu +1}^{n_{\mathrm{ar}}}exp\left(-\frac{1}{2s^2}{({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}},$$

(15)

and the $m(m+1)/2$ dimensional vector ${\mathsf{\zeta }}^j$ collects all the entries of the upper triangular part of the matrix logarithm $[log{C}^j]$ of the matrix $\left[C^j\right]$, defined as

$$\left[C^j\right]={\displaystyle \frac{{\sum }_{\mathsf{\ell }=\nu +1}^{n_{\mathrm{ar}}}{\tilde{\mathsf{\eta }}}_{\mathrm{ar}}^{\mathsf{\ell }}\left({\mathbf{w}}_{\mathrm{ar}}^j\right){\left({\tilde{\mathsf{\eta }}}_{\mathrm{ar}}^{\mathsf{\ell }}\left({\mathbf{w}}_{\mathrm{ar}}^j\right)\right)}^{T}exp\left(-\frac{1}{2s^2}{({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}{{\sum }_{\mathsf{\ell }=\nu +1}^{n_{\mathrm{ar}}}exp\left(-\frac{1}{2s^2}{({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})}^T{\left[C_{\mathbf{W}}\right]}^{-1}({\mathbf{w}}_{\mathrm{ar}}^j-{\mathbf{w}}_{\mathrm{ar}}^{\mathsf{\ell }})\right)}}.$$

(16)

Note that $n_{\mathrm{ar}}-\nu$ is the number of realizations used for the GKDE-based estimates of the conditional mean and conditional covariance of $\mathbf{H}$, given $\mathbf{W}={\mathbf{w}}_{\mathrm{ar}}^j$. Hence, the least-square estimation of parameters ${\mathsf{\theta }}_1$ and ${\mathsf{\theta }}_2$ is obtained as the parameters that minimize the cost function $\mathcal{J}({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$, written as

$$\mathcal{J}({\mathsf{\theta }}_1,{\mathsf{\theta }}_2)=\sum _{j=1}^{\nu }\parallel {\mathsf{\mu }}^j-{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}({\mathbf{w}}_{\mathrm{ar}}^j;{\mathsf{\theta }}_1){\parallel }^2+\sum _{j=1}^{\nu }{\parallel {\mathsf{\zeta }}^j-{\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}({\mathbf{w}}_{\mathrm{ar}}^j;{\mathsf{\theta }}_2)\parallel }^2,$$

(17)

This optimization problem is solved using the ADAM (ADAptive Moment estimation) algorithm [24] with a learning rate $\gamma$ scheduler that adjusts the learning rate over the course of training as

$$\gamma \left(k\right)=\max ({\gamma }_o{\alpha }^{(k-1)/\Delta },{\gamma }_{\min }),$$

(18)

where k is the epoch, ${\gamma }_o$ is the initial learning rate (default value is ${\gamma }_o=0.001$), $\alpha =0.95$ is the decay factor, $\Delta =5$ is the decay period, and ${\gamma }_{\min }=1\times {10}^{-7}$ is the minimum leaning rate. For the ADAM algorithm, the parameters are fixed as ${\beta }_1=0.9$, ${\beta }_2=0.99$, and $ϵ=1\times {10}^{-8}$.

5. Numerical Applications

5.1. Architecture of the Statistical ANN-Based Metamodel

Concerning the architecture of the statistical ANN-based metamodel, rather than constructing two multi-outputs fully connected feedforward networks (one for ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1)$ and one for ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)$), we choose to construct m single output fully connected feedforward networks ${\left\{{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_{1,1})\right\}}_1,\dots ,{\left\{{\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_{1,m})\right\}}_m$ and $d=m(m+1)/2$ single output fully connected feedforward networks ${\left\{{\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_{2,1})\right\}}_1,\dots ,{\left\{{\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_{2,d})\right\}}_d$, where ${\mathsf{\theta }}_1$ is rewritten as ${\mathsf{\theta }}_1=({\mathsf{\theta }}_{1,1},\dots ,{\mathsf{\theta }}_{1,m})$ and ${\mathsf{\theta }}_2$ is rewritten as ${\mathsf{\theta }}_2=({\mathsf{\theta }}_{2,1},\dots ,{\mathsf{\theta }}_{2,d})$. The training is carried out in parallel on a cluster of three Tesla V100 GPUs (Nvidia, Santa Clara, CA, USA). For each fully connected feedforward network, there are four hidden layers, and the number of units is 20, 250, 75 and 25, respectively. This architecture implies a total number of 26,314 parameters (biases and weights). For the first and the fourth layer, the Glorot [25] initialization is used; for the second and third layer, the He [26] initialization is used. Rectified linear unit (ReLU) activation functions are used for each of the four hidden layers.

5.2. Statistical Convergence Analysis for the Learned GKDE-Based Estimates’ Dataset

Statistical convergence analysis is carried out with respect to number $\nu$ (see Section 4.3). Figure 3 shows the graph of $n_{\mathrm{ar}}\mapsto {(n_{\mathrm{ar}}-\nu +1)}^{-1}{\sum }_{j=\nu +1}^{n_{\mathrm{ar}}}{\parallel \left[C_{\mathbf{H}|\mathbf{W}}\left({\mathbf{w}}_{\mathrm{ar}}^j\right)\right]\parallel }^2$ in which $\parallel ·\parallel$ is the Frobenius norm. The horizontal axis of Figure 3 is the number $n_{\mathrm{ar}}-\nu$, which comprises statistically independent realizations used in order to construct the GKDE-based estimates of conditional covariance of $\mathbf{H}$, given $\mathbf{W}=\mathbf{w}$ with $\nu$ = 60,000. It can be shown that convergence is reached for $n_{\mathrm{ar}}-\nu$ = 200,000; thus, $n_{\mathrm{ar}}$ = 260,000.

5.3. Conditional Covariance Matrices of $\mathit{R}$ and $\mathit{V}$ Given $\mathit{W}$

Using the database ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$ and conditional statistics based on the GKDE, the $(m\times m)$ conditional covariance matrix $\left[C_{\mathbf{Q}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ can be calculated. The expression is not included in this paper but is similar to Equation (3) using ${\mathbf{q}}_{\mathrm{ar}}^j$ instead of ${\mathsf{\eta }}_{\mathrm{ar}}^j$ and with a Silverman bandwidth $n=n_{\mathbf{q}}+n_{\mathbf{w}}$ in Equation (4). Using Equations (7)–(10) allow for calculating conditional covariance matrices of resistance $\left[C_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ and reactance $\left[C_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ for any given $\mathbf{W}=\mathbf{w}$. Furthermore, as explained in Section 3.3 and given models ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1)$ and ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)$, the conditional covariance matrix $\left[C_{\mathbf{R}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2)\right]$ (resp. $\left[C_{\mathbf{V}|\mathbf{W}}(\mathbf{w},{\mathsf{\theta }}_2)\right]$) of $\mathbf{R}\left(\mathbf{w}\right)$ (resp. $\mathbf{V}\left(\mathbf{w}\right)$) can be constructed for any given $\mathbf{W}=\mathbf{w}$. The entries of matrices $\left[C_{\mathbf{R}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ and $\left[C_{\mathbf{V}|\mathbf{W}}\left(\mathbf{w}\right)\right]$ (calculated with the GKDE) are displayed in Figure 4. The entries of matrices $\left[C_{\mathbf{R}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2)\right]$ and $\left[C_{\mathbf{V}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)\right]$ (calculated using the statistical ANN-based metamodel) are displayed in Figure 5. The frequency-sampled resistance is displayed in Figure 4a–d and Figure 5a–d. The frequency-sampled reactance is displayed in Figure 4e–h and Figure 5e–h. It should be noted that the considered values of the control parameters are indicated in each sub-figure. It can be seen that the matrices are not diagonal and that a correlation exits between the different frequency points of the frequency-sampled impedance. It can also be seen that the GKDE-based estimate in Figure 4 and statistical ANN-based metamodel estimate in Figure 5 of the covariance matrices are quantitatively the same.

5.4. Frequency-Sampled Impedance Using the Statistical ANN-Based Metamodel

For any value of $\mathbf{w}$ in its admissible set, the conditional statistics (mean values and confidence regions) of $\mathbf{R}\left(\mathbf{w}\right)$ and $\mathbf{V}\left(\mathbf{w}\right)$ are estimated using the statistically independent realizations ${\mathbf{r}}^1(\mathbf{w};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2),$$\dots ,$${\mathbf{r}}^N(\mathbf{w};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$ and ${\mathbf{v}}^1(\mathbf{w};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2),\dots ,{\mathbf{v}}^N(\mathbf{w};{\mathsf{\theta }}_1,{\mathsf{\theta }}_2)$, as presented in Section 3.3, and obtained using ${\mathsf{\mu }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_1)$ and ${\mathsf{\zeta }}_{\mathbf{H}|\mathbf{W}}(\mathbf{w};{\mathsf{\theta }}_2)$, as explained in Section 4.1.

Figure 6 shows 10 realizations of frequency-sampled resistances (Figure 6a–d) and 10 corresponding realizations of frequency-sampled reactance (the values of the control parameters are indicated in each sub-figure). The dashed blue line (resp. the blue domain) is the conditional mean value (resp. the conditional confidence region) of the frequency-sampled resistance (Figure 6a–d) and reactance (Figure 6a–d).

The conditional mean value and the conditional confidence region with probability level $P_c=98\%$, of the frequency-sampled impedance given $\mathbf{W}=\mathbf{w}$, corresponding to the statistical ANN-based metamodel, are presented in Figure 7 and are compared with the results presented in [18] (GKDE-based estimates). Figure 7a–d correspond to the resistance, and Figure 7e–h correspond to the reactance (the values of the control parameters are indicated in each sub-figure). The red dashed line (resp. the red domain) shows the conditional mean value (resp. the confidence region) from [18]. The blue dashed line (resp. the blue domain) shows the conditional mean value (resp. confidence region) from the statistical ANN-based metamodel. It can be seen that, concerning the conditional mean values of the frequency-sampled impedance, there is a good match between the statistical ANN-based metamodel presented in this paper and the previous results presented in [18]. The statistical ANN-based metamodel, which was fitted on the learned GKDE-based estimates’ dataset ${\mathcal{D}}_{\mathbf{H}|\mathbf{W}}^{\ast }$ (with 60,000 realizations), shows a conditional confidence region that is quantitatively the same as those calculated by the GKDE-based estimate using the learned ACM dataset ${\mathcal{D}}_{\mathrm{ar}}^{\ast }$ (260,000 realizations) with manually adjusted Silverman bandwidth s to the value $0.1186$.

6. Conclusions and Perspectives

In this paper, a statistical ANN-based metamodel of the frequency-sampled liner acoustic impedance was presented, for which only a small ACM dataset was available to fit its parameters. The control parameters were the POA and SPL, with the Mach number being kept at zero. The latent (unobserved) parameters introduce uncertainties that are taken into account by a probabilistic model introduced in [18]. The construction of the statistical metamodel used PCA to construct a statistically reduced representation of the frequency-sampled vector of the random log-resistance and the random reactance. For fitting the statistical ANN-based metamodel, a big training dataset was constructed using probabilistic learning on manifolds (PLoM). A priorconditional probability distribution of the reduced representation given the control parameters was then constructed and assumed to be Gaussian, yielding a multivariate log-normal distribution for the resistance and a multivariate Gaussian distribution for the reactance. The metamodels of the hyperparameters of such a conditional probabilistic model were presented as fully connected feedforward neural networks. As the reduced representation was modeled by a centered and normalized random vector, some constraints had to be taken into account in the minimization of the negative-log likelihood when fitting the parameters (biases and weights) of the neural network. The constrained problem was transformed into an unconstrained one, requiring the construction of a second training dataset to estimate the conditional mean vectors and conditional covariance matrices for which the learned realizations were generated using PLoM. The novelty of this paper lies in the methodology used to construct a statistical ANN-based metamodel of liner acoustic impedance, which can be used as a low-computational-cost metamodel to predict the confidence interval and the mean value of the impedance, given any value of the control parameter. The gradients of the mean values and confidence regions can also easily be derived using classical backpropagation algorithms for a very cheap computational cost. The statistical ANN-based metamodel presented in this paper is a complement to the model proposed in [18]. Firstly, it allows for offline use, as it does not require access to the learned dataset for making predictions. Secondly, the gradient computations with respect to input parameters are expected to be computationally cheaper, facilitating its use in optimization problems. These improvements, combined with the ANN’s ability to capture complex, multidimensional relationships, make this new model a flexible and efficient tool for optimizing liner acoustic impedance across the parameter space. Such work can be extended to a non-zero Mach number in the ACM dataset.