Overview of Arbitrarily High-Order Adjoint Sensitivity and Uncertainty Quantification Methodology for Large-Scale Systems

Abstract

This work reviews from a unified viewpoint the concepts underlying the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (n^th-CASAM-L) and the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (n^th-CASAM-N) methodologies. The practical application of the n^th-CASAM-L methodology is illustrated for an OECD/NEA reactor physics benchmark, while the practical application of the n^th-CASAM-N methodology is illustrated for a nonlinear model of reactor dynamics that exhibits periodic and chaotic oscillations. As illustrated both by the general theory and by the examples reviewed in this work, both the n^th-CASAM-L and n^th-CASAM-N methodologies overcome the curse of dimensionality in sensitivity analysis. The availability of efficiently and exactly computed sensitivities of arbitrarily high order can lead to major advances in all areas that need such high-order sensitivities, including data assimilation, model calibration, uncertainty reduction, and predictive modeling.

1. Introduction

The computational model of a physical system comprises the following fundamental conceptual components: (a) a well-posed system of equations that relate the system’s independent variables and parameters to the system’s state (i.e., dependent variables); (b) nominal or mean values for the system parameters, along with information regarding the probability distributions, moments thereof, and inequality and/or equality constraints that define the range of variations in the system’s parameters; and (c) one or several quantities, customarily referred to as model/system responses (or objective functions, or indices of performance), which are computed using the mathematical model. The functional derivatives of model responses with respect to model parameters are called “sensitivities” (of responses with respect to model parameters). The computation of higher-order sensitivities by conventional methods is subject to the “curse of dimensionality”, a term coined by Belmann [1] to describe phenomena in which the number of computations increases exponentially in the respective phase space. In the particular case of sensitivity analysis, the number of large-scale computations increases exponentially in the parameter phase space as the order of sensitivities increases if conventional methods (e.g., finite differences) are used: thus, if a model has TP parameters, the computation of first-order sensitivities requires at least 2TP large-scale computations; the computation of second-order sensitivities requires at least (3TP)² computations, and so on; and the computation of the n^th-order sensitivities requires at least (nTP)ⁿ large-scale computations.

A fundamental breakthrough in the sensitivity analysis of problems of interest to nuclear reactor physics, which are governed by the linear Boltzmann transport equation, was achieved by Wigner [2], who pioneered the use of the adjoint Boltzmann transport equation to compute first-order perturbations (and, hence, first-order sensitivities) in the reactor’s multiplication factor arising from perturbations in the model parameters (nuclear cross sections). Wigner’s method became widespread in the nuclear engineering community and was considered to be adequate for predicting the effects of small parameter changes on the model responses, as well as for performing [3,4] a first-order uncertainty quantification of the standard deviation induced in a model response due to uncertainties (standard deviations and correlations) affecting the model parameters (nuclear cross sections). This situation changed after Goldstein [5] reported studies of neutron transmission through a sodium shield, which showed that even for modest variations of 5% in the cross section at the 297 keV sodium minimum, the first-order sensitivities of responses measured by detectors differed by 20–50% from their correct/actual values. These results indicated that the nonlinear dependence of a detector’s response on the sodium cross section in deep-penetration radiation and neutron shielding problems cannot be predicted by first-order sensitivities, despite the fact that the physics of neutron/gamma transport through materials is governed by the linear Boltzmann equation. These findings have motivated Greenspan et al. [6] to propose an extension of Wigner’s method [2], so as to enable the computation of second-order sensitivities of detector responses to neutron cross sections for linear neutron transport problems. Greenspan et al. [6] concluded that their methodology “appears to provide satisfactory accuracies for most non-linear problems associated with cross section changes encountered in practice”. This conclusion, while correct for the particular “neutron-transmission through a sodium shield problem” under investigation, is actually incorrect in general since, as is shown in the remainder of this work, third- and fourth-order sensitivities of responses with respect to cross sections are even more important than the second-order sensitivities in reactor physics problems. Greenspan et al. [6] further stated that: “The use of higher (than second) order sensitivity theories like the one recently proposed by Gandini [7]—is not likely to be desirable—the added accuracy provided, say, by a third order formulation is not likely to justify the additional computational effort required for its application”. Again, this statement by Greenspan et al. [6] is correct regarding the methodology proposed by Gandini [7], which is not only cumbersome and computationally inefficient but is also limited to formal time-dependent linear neutron transport and/or diffusion problems. However, the sweepingly general statement by Greenspan et al. [6] that the “use of higher (than second) order sensitivity theories…is not likely to be desirable” is proven to be incorrect in the remainder of this work by showing that the third- and fourth-order sensitivities of responses with respect to cross sections are even more important than the second-order sensitivities in reactor physics problems.

Wigner’s methodology and its extensions by Greenspan et al. [6] and Gandini [7] were developed within the specific context of the linear neutron transport equation and are not applicable to nonlinear models, since nonlinear operators do not admit adjoint operators. The rigorous first-order adjoint sensitivity analysis methodology for generic large-scale nonlinear (as opposed to linearized) systems involving generic operator responses has been conceived and developed by Cacuci [8,9], who is also credited (see, e.g., [10,11]) for having introduced these principles to the earth, atmospheric, and other sciences. Cacuci’s [8,9] first-order adjoint sensitivity analysis methodology for nonlinear systems is just as efficient for computing first-order sensitivities as Wigner’s method is for linear systems, requiring a single large-scale (adjoint) computation for obtaining all of the first-order sensitivities, regardless of the number of model parameters. This is in contradistinction with the number of (at least) “2P computations” which would be required by conventional methods (e.g., finite differences). Thus, the methodologies of Wigner [2] and Cacuci [8,9] provided the “first-order cracking of the curse of dimensionality” in sensitivity and uncertainty analysis. Cacuci’s methodology [8,9] has been applied to many (too numerous to cite here) research fields since its publication. Second-order deterministic analyses of specific problems in diverse fields, e.g., structural analysis [12,13], mechanical system [14], atmospheric science [15], linear circuit analysis, and optimization [16], have also been published. General methodologies for the efficient and exact computation of second-order response sensitivities to model responses were proposed by Cacuci for response-coupled forward and adjoint linear systems [17] and for nonlinear systems [18].

The unparalleled efficiency of the second-order adjoint sensitivity analysis methodology for linear systems conceived by Cacuci [17] was demonstrated in [19,20,21,22,23,24] by applying this methodology to compute exactly the 21,976 first-order sensitivities and 482,944,576 s-order sensitivities for an OECD/NEA reactor physics benchmark [25], which is representative of a large-scale system that involves many parameters. The results obtained in [19,20,21,22,23,24] indicated that many second-order sensitivities were much larger than the largest first-order ones, which implies that the consideration of only the first-order sensitivities is insufficient for making credible predictions regarding the expected values and uncertainties (variances, covariances, skewness) of calculated and predicted/adjusted responses. At the very least, the second-order sensitivities must also be computed in order to enable the quantitative assessment of their impact on the predicted model responses.

The findings reported in [19,20,21,22,23,24] motivated the investigation [26,27] of the largest third-order sensitivities, many of which were found to be even larger than the second-order ones. This finding, in turn, has motivated the development [28] of the mathematical framework for determining and computing the fourth-order sensitivities for the OECD/NEA benchmark, many of which were found [29,30] to be larger than the third-order ones. The need for computing the third- and fourth-order sensitivities has been underscored in [31,32]. The efficiency of Cacuci’s new methodology for computing exactly third-order and fourth-order sensitivities while overcoming the curse of dimensionality has been highlighted in [33,34,35].

This sequence of findings has motivated the conception by Cacuci [36] of the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (abbreviated as “n^th-CASAM-L”). The n^th-CASAM-L enables the efficient computation of exactly determined expressions of arbitrarily high-order sensitivities of a generic system response—which can depend on both the forward and adjoint state functions− with respect to all of the parameters that characterize the physical system. The qualifier “comprehensive” is employed in order to highlight that the model parameters considered within the framework of the n^th-CASAM-L include the system’s uncertain boundaries and internal interfaces in the phase space. The n^th-CASAM-L mathematical framework was developed specifically for linear systems because the most important model responses produced by such systems are various Lagrangian functionals, which depend simultaneously on both the forward and adjoint state functions governing the respective linear system. Included among such functionals are the Raleigh quotient for computing eigenvalues and/or separation constants when solving linear partial differential equations, and the Schwinger and Rousopoulos functionals (see, e.g., [37,38,39]), which play fundamental roles in optimization and control procedures, derivation of numerical methods for solving equations (differential, integral, integro-differential), and sensitivity analysis. Responses that depend on both the forward and adjoint state functions underlying the respective model can occur only for linear systems, because responses in nonlinear systems can only depend on the system’s forward state functions (since nonlinear operators do not admit adjoint operators). Consequently, the sensitivity analysis of responses that simultaneously involve both forward and adjoint state functions makes it necessary to treat linear models/systems in their own right, rather than treating them as particular cases of nonlinear systems. The n^th-CASAM-L is formulated in linearly increasing higher-dimensional Hilbert spaces (as opposed to exponentially increasing parameter-dimensional spaces), thus overcoming the curse of dimensionality in enabling the most efficient computation of exactly determined expressions of arbitrarily high-order sensitivities of generic nonlinear system responses with respect to model parameters, uncertain boundaries, and internal interfaces in the model’s phase space.

In parallel with the aforementioned developments, Cacuci [40,41] has extended his original work [8,9] on nonlinear systems, providing the mathematical framework for deriving and computing efficiently the exact expressions of sensitivities with respect to imprecisely known (i.e., uncertain) model parameters, model boundaries, and/or internal interfaces, up to and including fifth order. Subsequently, Cacuci [42] has generalized the 5^th-CASAM-N methodology [41] to enable the exact and efficient computation of sensitivities of arbitrarily high order with respect to model parameters, uncertain boundaries, and internal interfaces in the model’s phase space. This novel general methodology [42], called the n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (n^th-CASAM-N), is also formulated (just like the n^th-CASAM-L) in linearly increasing higher-dimensional Hilbert spaces (as opposed to exponentially increasing parameter-dimensional spaces), thus overcoming the curse of dimensionality in sensitivity analysis of nonlinear systems.

This work is structured as follows: Section 2 highlights the fundamental role of sensitivities in defining the correlations between responses and parameters. If the negative impact of the curse of dimensionality is not alleviated when computing sensitivities, this negative impact is transmitted to the computation of the aforementioned correlations. Section 3.1 illustrates the need for computing high-order sensitivities by reviewing the results obtained in [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30] for an OECD/NEA reactor physics benchmark [25], which is representative of linear models/systems. These results underscore the efficiency of the n^th-CASAM-L in overcoming the curse of dimensionality while producing the respective results. Section 3.2 reviews the general mathematical framework underlying the n^th-CASAM-L [36].

Section 4.1 reviews the pioneering sensitivity and uncertainty analysis results presented in [43,44] for the reduced-order model developed by March-Leuba, Cacuci, and Perez [45] for studying the nonlinear dynamics displayed by boiling water reactors (BWR). This reduced-order model comprises point neutron kinetics equations coupled nonlinearly to thermal-hydraulics equations that describe the time evolution of the fuel temperature and coolant density in the recirculation loop, which is treated as a single path of fluid of variable cross-sectional area but with constant mass flow, with the coolant entering the core at saturation temperature. This model shows that the major state variables, including power, reactivity, void fraction, and temperature, undergo limit cycle oscillations under high-power/low-flow operating conditions, and the amplitudes of these oscillations are very sensitive to the reactor’s operating conditions. When increasing the heat transfer from the reactor to the coolant, this model indicates that these limit cycle oscillations can become unstable, undergoing period-doubling bifurcations in the time evolutions of the state variables (power, delayed neutron precursors, reactivity, coolant void fraction, fuel temperature) from periodic to aperiodic (chaotic) oscillations. Thus, this nonlinear model [45] illustrates generically the most extreme forms of nonlinear phenomena (from limit cycles to chaos) that can be displayed by nonlinear physical systems. The results of the sensitivity and uncertainty analyses presented in Section 4.1 are obtained by applying the first-order particularization of the n^th-CASAM-N [42], the general form of which is reviewed in Section 4.2. The work of March-Leuba, Cacuci, and Perez [45] has inspired several related models of BWR-dynamics [46,47,48,49,50,51,52,53,54,55].

Section 5 reviews the concepts underlying the n^th-CASAM-L [36,56] and n^th-CASAM-N [42] methodologies from a unified viewpoint, indicating under which circumstances the n^th-CASAM-L could be viewed as a particular case of the n^th-CASAM-N methodology. As the paradigm examples presented in Section 3.1 and Section 4.1 illustrate, using finite differences for computing high-order sensitivities of model responses to model parameters becomes not only computationally unfeasible for large-scale systems (because of the curse of dimensionality), but finding the optimal step size to minimize the error between the finite difference result and the exact result is practically impossible to achieve unless one knows beforehand what the exact result is—which would be possible only by using the CASAM-L and/or n^th-CASAM-N methodologies.

2. The Fundamental Role of High-Order Sensitivities in Defining Correlations among Model Responses and Parameters

The mathematical model of a physical system comprises independent variables (e.g., space, time, etc.), dependent variables (aka “state functions”; e.g., temperature, mass, momentum, etc.), and various parameters (appearing in correlations, coordinates of physical boundaries, etc.), which are all interrelated by equations that usually represent conservation laws. Without loss of generality, the model parameters can be considered to be real scalar quantities, having known nominal (or mean) values and, possibly, known higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis), which are usually determined from experimental data and/or processes external to the physical system under consideration. The model parameters usually stem from processes that are external to the system under consideration and are seldom, if ever, known precisely. These imprecisely known model parameters are denoted as ${\mathsf{\alpha }}_1$,…, ${\mathsf{\alpha }}_{TP}$, where the subscript “TP” indicates “Total (number of) Parameters”. It is convenient to consider that these parameters are components of a “vector of parameters”, denoted as $\mathsf{\alpha }\triangleq {\left({\mathsf{\alpha }}_1,\dots ,{\mathsf{\alpha }}_{TP}\right)}^{†}\in {ℝ}^{TP}$, where ${ℝ}^{TP}$ denotes the TP-dimensional subset of the set of real scalars. These model parameters are considered to include imprecisely known geometrical parameters that characterize the physical system’s boundaries in the phase space of the model’s independent variables. Thus, $\mathsf{\alpha }\in {ℝ}^{TP}$ is a TP-dimensional column vector and its components are considered to include all of the uncertain model parameters, including those that may enter into defining the system’s boundary in the phase space of independent variables. For subsequent developments, it is convenient to denote matrices and vectors using capital and lower case bold letters, respectively. The symbol “$\triangleq$” will be used to denote “is defined as” or “is by definition equal to”. Transposition will be indicated by a dagger $(†)$ superscript.

The result of interest computing using a mathematical/computational model of a physical system is, in principle, an implicit function of the system’s parameters $\mathsf{\alpha }\triangleq {\left({\mathsf{\alpha }}_1,\dots ,{\mathsf{\alpha }}_{TP}\right)}^{†}\in {ℝ}^{TP}$. In practice, the model parameters are not known exactly, even though they are not bona fide random quantities. For practical purposes, however, these model parameters are considered to be variates that obey a multivariate probability distribution function, denoted as $p_{\mathsf{\alpha }}\left(\mathsf{\alpha }\right)$, which is seldom known, particularly for large-scale systems involving many parameters. Considering that the multivariate distribution $p_{\mathsf{\alpha }}\left(\mathsf{\alpha }\right)$ is formally defined on a domain $D_{\mathsf{\alpha }}$, the various moments of $p_{\mathsf{\alpha }}\left(\mathsf{\alpha }\right)$ can be formally defined in a standard manner by using the following notation

$${\langle u\left(\mathsf{\alpha }\right)\rangle }_{\mathsf{\alpha }}\triangleq {\displaystyle \underset{D_{\mathsf{\alpha }}}{\int }u\left(\mathsf{\alpha }\right)p_{\mathsf{\alpha }}\left(\mathsf{\alpha }\right)d\mathsf{\alpha }}$$

(1)

where $u\left(\mathsf{\alpha }\right)$ is a continuous function of the parameters $\mathsf{\alpha }$. Thus, the moments of $p_{\mathsf{\alpha }}\left(\mathsf{\alpha }\right)$ are defined as follows:

• The expected (or mean) value of a model parameter ${\mathsf{\alpha }}_j$, denoted as ${\mathsf{\alpha }}_j^0$, is defined as follows:

  $${\mathsf{\alpha }}_j^0\triangleq {\langle {\mathsf{\alpha }}_j\rangle }_{\mathsf{\alpha }}\triangleq {\displaystyle \underset{D_{\mathsf{\alpha }}}{\int }{\mathsf{\alpha }}_j{p}_{\mathsf{\alpha }}\left(\mathsf{\alpha }\right)d\mathsf{\alpha }}, j=1,\dots ,TP.$$

  (2)

  The expected values ${\mathsf{\alpha }}_j^0$ are considered to be the components of the following vector of mean (expected) values:

  $${\mathsf{\alpha }}^0\triangleq {\left({\mathsf{\alpha }}_1^0,\dots ,{\mathsf{\alpha }}_{TP}^0\right)}^{†}$$

  (3)
• The covariance, $\mathrm{cov}\left({\mathsf{\alpha }}_{j_1},{\mathsf{\alpha }}_{j_2}\right)$, of two parameters, ${\mathsf{\alpha }}_{j_1}$ and ${\mathsf{\alpha }}_{j_2}$, is defined as follows:

  $$\mathrm{cov}\left({\mathsf{\alpha }}_{j_1},{\mathsf{\alpha }}_{j_2}\right)\triangleq {\langle \left({\mathsf{\alpha }}_{j_1}-{\mathsf{\alpha }}_{j_1}^0\right)\left({\mathsf{\alpha }}_{j_2}-{\mathsf{\alpha }}_{j_2}^0\right)\rangle }_{\mathsf{\alpha }}, j_1,j_2=1,\dots ,TP.$$

  (4)

  The variance, $\mathrm{var}\left({\mathsf{\alpha }}_{j_1}\right)$, of a parameter ${\mathsf{\alpha }}_{j_1}$, is defined as follows:

  $$\mathrm{var}\left({\mathsf{\alpha }}_{j_1}\right)\triangleq {\langle {\left({\mathsf{\alpha }}_{j_1}-{\mathsf{\alpha }}_{j_1}^0\right)}^2\rangle }_{\mathsf{\alpha }}, j_1=1,\dots ,TP.$$

  (5)

  The standard deviation of a parameter ${\mathsf{\alpha }}_{j_1}$, which is denoted ${\mathsf{\sigma }}_{j_1}$, is defined as follows: ${\mathsf{\sigma }}_j\triangleq \sqrt{\mathrm{var}({\mathsf{\alpha }}_j)}$. The correlation, ${\mathsf{\rho }}_{j_1{j}_2}$, between two parameters ${\mathsf{\alpha }}_{j_1}$ and ${\mathsf{\alpha }}_{j_2}$, is defined as follows:

  $${\mathsf{\rho }}_{j_1{j}_2}\triangleq \mathrm{cov}\left({\mathsf{\alpha }}_{j_1},{\mathsf{\alpha }}_{j_2}\right)/\left({\mathsf{\sigma }}_{j_1}{\mathsf{\sigma }}_{j_2}\right); j_1,j_2=1,\dots ,TP.$$

  (6)
• The third-order correlation, $t_{j_1{j}_2{j}_3}$, among three parameters ${\mathsf{\alpha }}_{j_1}$, ${\mathsf{\alpha }}_{j_2}$ and ${\mathsf{\alpha }}_{j_3}$, is defined as follows:

  $$t_{j_1{j}_2{j}_3}\triangleq {\langle \left({\mathsf{\alpha }}_{j_1}-{\mathsf{\alpha }}_{j_1}^0\right)\left({\mathsf{\alpha }}_{j_2}-{\mathsf{\alpha }}_{j_2}^0\right)\left({\mathsf{\alpha }}_{j_3}-{\mathsf{\alpha }}_{j_3}^0\right)\rangle }_{\mathsf{\alpha }}/\left({\mathsf{\sigma }}_{j_1}{\mathsf{\sigma }}_{j_2}{\mathsf{\sigma }}_{j_3}\right); j_1,j_2,j_3=1,\dots ,TP.$$

  (7)
• The fourth-order correlation $q_{j1j2j3j4}$ among four parameters, is defined as follows, for $j_1,j_2,j_3,j_4=1, \dots , TP$:

  $$q_{j_1{j}_2{j}_3{j}_3}\triangleq \frac{{\langle \left({\mathsf{\alpha }}_{j_1}-{\mathsf{\alpha }}_{j_1}^0\right)\left({\mathsf{\alpha }}_{j_2}-{\mathsf{\alpha }}_{j_2}^0\right)\left({\mathsf{\alpha }}_{j_3}-{\mathsf{\alpha }}_{j_3}^0\right)\left({\mathsf{\alpha }}_{j_4}-{\mathsf{\alpha }}_{j_4}^0\right)\rangle }_{\mathsf{\alpha }}}{{\mathsf{\sigma }}_{j_1}{\mathsf{\sigma }}_{j_2}{\mathsf{\sigma }}_{j_3}{\mathsf{\sigma }}_{j_4}}.$$

  (8)

  In the important particular case when the parameters are normally distributed, the following relation holds:

  $$q_{j_1{j}_2{j}_3{j}_3}={\mathsf{\rho }}_{j_1{j}_2}{\mathsf{\rho }}_{j_3{j}_4}+{\mathsf{\rho }}_{j_1{j}_3}{\mathsf{\rho }}_{j_2{j}_4}+{\mathsf{\rho }}_{j_1{j}_4}{\mathsf{\rho }}_{j_2{j}_3}.$$

  (9)

The higher-order correlations among parameters are defined following the same pattern as displayed in Equations (6)–(8). In practice, the parameter correlations are estimated using experimental information about the parameters.

The result of interest computed using the model of the physical system under consideration is customarily called a system/model “response” and is denoted as $R_k\left(\mathsf{\alpha }\right)$. Being an implicit function of the system’s parameters, the response is considered to admit a formal Taylor series of around a point ${\mathsf{\alpha }}^0\triangleq {\left({\mathsf{\alpha }}_1^0,\dots ,{\mathsf{\alpha }}_{TP}^0\right)}^{†}$, which is usually taken to be the vector of expected (or nominal) parameter values, having the following well-known formal expression:

$$\begin{array}{l}{R}_k\left(\mathsf{\alpha }\right)=R_k^{\left(0\right)}\left({\mathsf{\alpha }}^0\right)+{{\displaystyle \sum _{n=1}^{\infty }\frac{1}{n!}{\displaystyle \sum _{j_1=1}^{TP}{\displaystyle \sum _{j_2=1}^{TP}\dots }}{\displaystyle \sum _{j_{n-1}=1}^{TP}{\displaystyle \sum _{j_n=1}^{TP}\left\{\frac{{\partial }^n{R}_k\left(\mathsf{\alpha }\right)}{\partial {\mathsf{\alpha }}_{j_1}\partial {\mathsf{\alpha }}_{j_2}\dots \partial {\mathsf{\alpha }}_{j_{n-1}}\partial {\mathsf{\alpha }}_{j_n}}\right\}}}}}_{\mathsf{\alpha }={\mathsf{\alpha }}^0}\\ \times \left[\left({\mathsf{\alpha }}_{j_1}-{\mathsf{\alpha }}_{j_1}^0\right)\left({\mathsf{\alpha }}_{j_2}-{\mathsf{\alpha }}_{j_2}^0\right)\dots \left({\mathsf{\alpha }}_{j_{n-1}}-{\mathsf{\alpha }}_{j_{4n-1}}^0\right)\left({\mathsf{\alpha }}_{j_n}-{\mathsf{\alpha }}_{j_n}^0\right)\right].\end{array}$$

(10)

The radius/domain of convergence of the series in Equation (10) must be established before using it for any practical application. The functional derivatives of the model response $R_k\left(\mathsf{\alpha }\right)$ with respect to the model’s parameters are customarily called the “sensitivities” (of various orders) of the model’s response with respect to the parameters. As is well known, and as indicated by Equation (10), the Taylor series of a function of $TP$ variables (e.g., $R_k\left(\mathsf{\alpha }\right)$) comprises $TP$ first-order derivatives, $TP\left(TP+1\right)/2$ distinct second-order derivatives, $TP\left(TP+1\right)\left(TP+2\right)/6$ distinct third-order derivatives, and so on. The computation by conventional methods of the $n^{th}$-order sensitivities of a response with respect to the $TP$ parameters would require at least $O\left(T{P}^n\right)$ large-scale computations. This exponential increase—with the order of response sensitivities—in the number of large-scale computations needed to determine higher-order sensitivities is the manifestation of the “curse of dimensionality in sensitivity analysis”, by analogy to expression coined by Belmann [1] to express the difficulty of using “brute-force” grid search when optimizing a function with many input variables.

It follows from the Taylor series expansion presented in Equation (10) that, in principle, the response and parameters are correlated quantities. The correlations of various orders between a model’s response(s) and parameters are defined as follows.

• The expectation (value), $E\left(R_k\right)$, of a response $R_k\left(\mathsf{\alpha }\right)$ is defined as follows:

  $$E\left(R_k\right)\triangleq {\langle R_k\left(\mathsf{\alpha }\right)\rangle }_{\mathsf{\alpha }}\triangleq {\displaystyle \underset{D_{\mathsf{\alpha }}}{\int }{R}_k\left(\mathsf{\alpha }\right)p_{\mathsf{\alpha }}\left(\mathsf{\alpha }\right)d\mathsf{\alpha }}$$

  (11)
• The covariance between two responses $R_k$ and $R_{\ell }$ is defined as follows:

  $$cov\left(R_k, R_{\ell }\right)\triangleq {\langle \left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\left[R_{\ell }\left(\mathsf{\alpha }\right)-E\left(R_{\ell }\right)\right]\rangle }_{\mathsf{\alpha }}.$$

  (12)
• The triple correlations (or third-order moment of the distribution of responses), denoted as ${\mathsf{\mu }}_3\left(R_k,R_{\ell },R_m\right)$, among three responses $R_k\left(\mathsf{\alpha }\right)$, $R_{\ell }\left(\mathsf{\alpha }\right)$, and $R_m\left(\mathsf{\alpha }\right)$, are defined as follows:

  $$\begin{array}{l}{\mathsf{\mu }}_3\left(R_k,R_{\ell },R_m\right)\\ \triangleq {\langle \left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\left[R_{\ell }\left(\mathsf{\alpha }\right)-E\left(R_{\ell }\right)\right]\left[R_m\left(\mathsf{\alpha }\right)-E\left(R_m\right)\right]\rangle }_{\mathsf{\alpha }}\end{array}$$

  (13)
• The quadruple correlations (or fourth-order moment of the distribution of responses), denoted as ${\mathsf{\mu }}_4\left(R_k,R_{\ell },R_m,R_n\right)$, among four responses $R_k\left(\mathsf{\alpha }\right)$, $R_{\ell }\left(\mathsf{\alpha }\right)$, $R_m\left(\mathsf{\alpha }\right)$, and $R_n\left(\mathsf{\alpha }\right)$, are defined as follows:

  $$\begin{array}{l}{\mathsf{\mu }}_4\left(R_k,R_{\ell },R_m,R_n\right)\triangleq \langle \left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\left[R_{\ell }\left(\mathsf{\alpha }\right)-E\left(R_{\ell }\right)\right]\\ {\times \left[R_m\left(\mathsf{\alpha }\right)-E\left(R_m\right)\right]\left[R_n\left(\mathsf{\alpha }\right)-E\left(R_n\right)\right]\rangle }_{\mathsf{\alpha }}.\end{array}$$

  (14)

Expressions for the response moments defined in Equations (11)−(14) were first obtained by Tuckey [57], who provided expressions up to fifth order in the parameter standard deviations. Cacuci [56] has generalized Tuckey’s work, providing corresponding expressions up to and including the sixth-order terms in parameter standard deviations. Cacuci [56] has also noticed that the existence of the Taylor-series shown in Equation (10) implies not only correlations among model responses but also correlations among model responses and model parameters, as defined below in Equations (15)−(20), and has provided the corresponding expressions up to and including the sixth-order terms in parameter standard deviations.

e.

The covariance between computed responses and a parameter is defined as follows:

$$\mathrm{cov}\left({\mathsf{\alpha }}_i,R_k\right)\triangleq {\langle \left({\mathsf{\alpha }}_i-{\mathsf{\alpha }}_i^0\right)\left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\rangle }_{\mathsf{\alpha }}.$$

(15)

f.

The triple-correlations among one parameter, ${\mathsf{\alpha }}_i$, and two responses, $R_k\left(\mathsf{\alpha }\right)$ and $R_{\ell }\left(\mathsf{\alpha }\right)$, are defined as follows:

$${\mathsf{\mu }}_3\left({\mathsf{\alpha }}_i,R_k,R_{\ell }\right)\triangleq {\langle \left({\mathsf{\alpha }}_i-{\mathsf{\alpha }}_i^0\right)\left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\left[R_{\ell }\left(\mathsf{\alpha }\right)-E\left(R_{\ell }\right)\right]\rangle }_{\mathsf{\alpha }}.$$

(16)

g.

The triple correlations among two parameters ${\mathsf{\alpha }}_i$, ${\mathsf{\alpha }}_j$ and one response $R_k\left(\mathsf{\alpha }\right)$ are defined as follows:

$${\mathsf{\mu }}_3\left({\mathsf{\alpha }}_i,{\mathsf{\alpha }}_j,R_k\right)\triangleq {\langle \left({\mathsf{\alpha }}_i-{\mathsf{\alpha }}_i^0\right)\left({\mathsf{\alpha }}_j-{\mathsf{\alpha }}_j^0\right)\left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\rangle }_{\mathsf{\alpha }}.$$

(17)

h.

The quadruple correlations denoted as ${\mathsf{\mu }}_4\left({\mathsf{\alpha }}_i,R_j,R_k,R_{\ell }\right)$, among one parameter, ${\mathsf{\alpha }}_i$, and three responses $R_j\left(\mathsf{\alpha }\right)$, $R_k\left(\mathsf{\alpha }\right)$, and $R_{\ell }\left(\mathsf{\alpha }\right)$, are defined as follows:

$${\mathsf{\mu }}_4\left({\mathsf{\alpha }}_i,R_j,R_k,R_{\ell }\right)\triangleq {\langle \left({\mathsf{\alpha }}_i-{\mathsf{\alpha }}_i^0\right)\left[R_j\left(\mathsf{\alpha }\right)-E\left(R_j\right)\right]\left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\left[R_{\ell }\left(\mathsf{\alpha }\right)-E\left(R_{\ell }\right)\right]\rangle }_{\mathsf{\alpha }}.$$

(18)

i.

The quadruple correlations, denoted as ${\mathsf{\mu }}_4\left({\mathsf{\alpha }}_i,{\mathsf{\alpha }}_j,R_k,R_{\ell }\right)$, among two parameters ${\mathsf{\alpha }}_i$, ${\mathsf{\alpha }}_j$ and two responses $R_k\left(\mathsf{\alpha }\right)$, $R_{\ell }\left(\mathsf{\alpha }\right)$, are defined as follows:

$${\mathsf{\mu }}_4\left({\mathsf{\alpha }}_i,{\mathsf{\alpha }}_j,R_k,R_{\ell }\right)\triangleq \langle \left({\mathsf{\alpha }}_i-{\mathsf{\alpha }}_i^0\right)\left({\mathsf{\alpha }}_j-{\mathsf{\alpha }}_j^0\right){\left[R_k\left(\mathsf{\alpha }\right)-E\left(R_k\right)\right]\left[R_{\ell }\left(\mathsf{\alpha }\right)-E\left(R_{\ell }\right)\right]\rangle }_{\mathsf{\alpha }}.$$

(19)

j.

The quadruple correlations, denoted as ${\mathsf{\mu }}_4\left({\mathsf{\alpha }}_i,{\mathsf{\alpha }}_j,{\mathsf{\alpha }}_k,R_{\ell }\right)$, among three parameters ${\mathsf{\alpha }}_i$, ${\mathsf{\alpha }}_j$, ${\mathsf{\alpha }}_k$ and one response, $R_{\ell }\left(\mathsf{\alpha }\right)$, are defined as follows:

$${\mathsf{\mu }}_4\left({\mathsf{\alpha }}_i,{\mathsf{\alpha }}_j,{\mathsf{\alpha }}_k,R_{\ell }\right)\triangleq {\langle \left({\mathsf{\alpha }}_i-{\mathsf{\alpha }}_i^0\right)\left({\mathsf{\alpha }}_j-{\mathsf{\alpha }}_j^0\right)\left({\mathsf{\alpha }}_k-{\mathsf{\alpha }}_k^0\right)\left[R_{\ell }\left(\mathsf{\alpha }\right)-E\left(R_{\ell }\right)\right]\rangle }_{\mathsf{\alpha }}.$$

(20)

3. High-Order Sensitivity and Uncertainty Analysis of Linear Models/Systems: Illustrative Example and General Theory

Section 3.1 illustrates the need for computing high-order sensitivities by reviewing the results obtained in [16,17,18,19,20,21,22,23,24,25,26,27,28,29,30] for an OECD/NEA reactor physics benchmark [25], comprising a subcritical polyethylene-reflected plutonium metal sphere, which is called the “PERP” (polyethylene-reflected plutonium) benchmark. The PERP benchmark is modeled by using the deterministic neutron transport Boltzmann equation, which is solved numerically (after discretization in the energy, spatial, and angular independent variables) using the software package PARTISN [58]. The numerical model of the PERP benchmark comprises 21,976 uncertain parameters, as follows: 180 group-averaged total microscopic cross sections, 21,600 group-averaged scattering microscopic cross sections, 120 fission process parameters, 60 fission spectrum parameters, 10 parameters describing the experiment’s nuclear sources, and 6 isotopic number densities. In view of the fact that the PERP benchmark is modeled by the integro-differential Boltzmann equation in six independent variables (the solving of which is representative of “large-scale” computations), and since this model involves 21,976 uncertain parameters (which are representative of “many parameters”), the PERP benchmark can be considered to be representative of linear models/systems for the purpose of computing high-order sensitivities.

The results reviewed in Section 3.1 were obtained by applying the n^th-CASAM-L, for n = 1, 2, 3, 4, underscoring the efficiency of the n^th-CASAM-L in overcoming the curse of dimensionality while producing the respective results. Section 3.2 reviews the general mathematical framework underlying the n^th-CASAM-L [36].

3.1. Illustrating the Need for High-Order Sensitivity Analysis and Uncertainty Quantification: A Paradigm Reactor Physics Benchmark

The PERP benchmark is a spherical subcritical nuclear system driven by a source of spontaneous fission neutrons. The PERP benchmark comprises an inner sphere (designated as “material 1” and assigned to “zone 1”) which is surrounded by a spherical shell (designated as “material 2” and assigned to “zone 2”). The inner sphere of the PERP benchmark contains α-phase plutonium which acts as the source of particles; the radius is $r_1$ = 3.794 cm. This inner sphere is surrounded by a spherical shell reflector made of polyethylene with a thickness of 3.81 cm; the radius of the outer shell containing polyethylene is $r_s$ = 7.604 cm.

Table 1. Dimensions and composition of the PERP benchmark.

Materials	Isotopes	Weight Fraction
Material 1 (plutonium metal)	Isotope 1 (239Pu)	9.3804 × 10−1
	Isotope 2 (240Pu)	5.9411 × 10−2
	Isotope 3 (69Ga)	1.5152 × 10−3
	Isotope 4 (71Ga)	1.0346 × 10−3
Material 2 (polyethylene)	Isotope 5 (12C)	8.5630 × 10−1
	Isotope 6 (1H)	1.4370 × 10−1

specifies the constitutive materials of the PERP benchmark.

The neutron flux distribution within the PERP benchmark has been computed by using the deterministic software package PARTISN [58], which solves the following multigroup approximation of the transport equation for the group fluxes, ${\mathsf{\phi }}^g\left(r,\mathsf{\Omega }\right)$:

$$B^g\left(\mathsf{\alpha }\right){\mathsf{\phi }}^g\left(r,\mathsf{\Omega }\right)=Q^g\left(\mathsf{\alpha };r\right), g=1,\dots ,G,$$

(21)

$${\mathsf{\phi }}^g\left(r_s,\mathsf{\Omega }\right)=0,r_s\in S_b,\mathsf{\Omega }·n<0, g=1,\dots ,G,$$

(22)

where:

$$\begin{array}{l}{B}^g\left(\mathsf{\alpha }\right){\mathsf{\phi }}^g\left(r,\mathsf{\Omega }\right)\triangleq \mathsf{\Omega }\mathbf{·}\nabla {\mathsf{\phi }}^g\left(r,\mathsf{\Omega }\right)+{\Sigma }_t^g\left(r\right) {\mathsf{\phi }}^g\left(r,\mathsf{\Omega }\right)\\ -{\displaystyle \sum _{g^{\prime }=1}^G{\displaystyle \underset{4\mathsf{\pi }}{\int }{\Sigma }_s^{g^{\prime }\to g}\left(r,{\mathsf{\Omega }}^{\prime }\mathbf{·}\mathsf{\Omega }\right) {\mathsf{\phi }}^{g^{\prime }}\left(r,{\mathsf{\Omega }}^{\prime }\right)d{\mathsf{\Omega }}^{\prime }} }\\ -{\mathsf{\chi }}^g\left(r\right){\displaystyle \sum _{g^{\prime }=1}^G{\displaystyle \underset{4\mathsf{\pi }}{\int }{\left(\mathsf{\nu }\Sigma \right)}_f^{g^{\prime }}\left(r\right) {\mathsf{\phi }}^{g^{\prime }}\left(r,{\mathsf{\Omega }}^{\prime }\right)d{\mathsf{\Omega }}^{\prime }} } ,\end{array}$$

(23)

$$Q^g\left(\mathsf{\alpha };r\right)\triangleq {\displaystyle \sum _{i=1}^{N_f}{\mathsf{\lambda }}_i{N}_{i,1}{F}_i^{SF}{\mathsf{\nu }}_i^{SF}\frac{1}{I_0}{\displaystyle {\int }_{E^{g+1}}^{E^g}dE e^{-E/a_i}\sinh \sqrt{b_{i}E}}}, g=1,\dots ,G;$$

(24)

$$I_0\triangleq \frac{\sqrt{\mathsf{\pi }{a}_i{}^3{b}_i}}{2}{e}^{\frac{a_i{b}_i}{4}}.$$

(25)

The PARTISN [58] computations used the SOURCE4C code [59] and the MENDF71X library [60] which comprise 618 group cross sections. These cross sections were collapsed to $G=30$ energy groups, with group boundaries, $E^g$, as indicated in Figure 1. The MENDF71X library [60] uses ENDF/B-VII.1 nuclear data [61]. The group boundaries, $E^g$, are user-defined and are therefore considered to be perfectly well-known parameters.

The fundamental quantities of interest (i.e., “system responses”) for subcritical benchmarks (such as the PERP benchmark) are single-counting rates, double-counting rates, the leakage multiplication, and the total leakage. The total leakage from the outer surface of the PERP sphere was considered to be the paradigm response of interest for illustrating the need for high-order sensitivity analysis, because the total leakage does not depend on the detector configuration (as opposed to count rates) and is therefore more meaningful physically than the count rates.

The total neutron leakage from the PERP sphere, denoted as $L\left(\mathsf{\alpha }\right)$, depends implicitly (through the neutron flux) on all model parameters and is defined as follows:

$$\begin{array}{l}L\left(\mathsf{\alpha }\right)\triangleq {\displaystyle \underset{V}{\int }dV}{\displaystyle \underset{0}{\overset{\infty }{\int }}dE}{\displaystyle \underset{\mathsf{\Omega }·n>0}{\int }d\mathsf{\Omega } \mathsf{\Omega }·n\mathsf{\delta }\left(r-r_s\right)} \mathsf{\phi }\left(r,E,\mathsf{\Omega }\right)\\ ={\displaystyle \underset{S_b}{\int }dS}{\displaystyle \sum _{g=1}^G{\displaystyle \underset{\mathsf{\Omega }·n>0}{\int }d\mathsf{\Omega }}} \mathsf{\Omega }·n \mathsf{\delta }\left(r-r_s\right){\mathsf{\phi }}^g\left(r,\mathsf{\Omega }\right).\end{array}$$

(26)

The numerical value of the PERP’s benchmark total leakage, computed by using Equation (26), is $1.7648\times {10}^6$ neutrons/sec. Figure 1 depicts the histogram plot of the leakage for each energy group for the PERP benchmark.

3.1.1. Overview of Results for the First, Second, Third, and Fourth-Order Sensitivities

The computations of the sensitivities, of all orders, were performed deliberately using a widely accessible desktop computer, namely a DELL AMD FX-8350 with an 8-core processor. A single adjoint (large-scale) computation is needed in order to obtain all 21,976 sensitives of the first-level adjoint function, which is subsequently used to compute all of the first-order sensitivities using inexpensive quadrature formulas. The CPU time for a typical PARTISN forward neutron transport computation with an angular quadrature of S₃₂ is ca. 45 s. On the other hand, the CPU time for a typical adjoint neutron transport computation using the neutron transport solver PARTISN with an angular quadrature of S₃₂ is ca. 24 s. The CPU time for computing the integrals over the adjoint function is ca. 0.004 s.

Thus, the exact computation of the 21,976 first-order sensitivities using the first-level adjoint function is (24 + 21,976 × 0.004) s = 112 s. By comparison, the approximate computation of the 21,976 first-order sensitivities using a two-point finite difference formula would require 21,976 × 2 × 45 s = 550 h. Most of the first-order relative sensitivities have been found to be negligibly small, having absolute values less than 0.1. The most important (i.e., largest) sensitivities of the leakage response are with respect to the group-averaged total microscopic cross sections, followed by the sensitivities of the leakage response with respect to the isotopic number densities. Only 16 first-order relative sensitivities have absolute values greater than 1.0; most of these involve the total cross sections of ¹H (hydrogen) and ²³⁹Pu (plutonium-239).

After having computed all of the first-order sensitivities and ranked their importance in the order of the magnitudes of the corresponding first-order relative sensitivities, it is imperative to compute the second-order sensitivities in order to quantify and compare their impact to the impact of the first-order sensitivities. The theoretical framework for computing efficiently and exactly second-order sensitivities has therefore been developed and applied to the PERP benchmark 2^nd-CASAM-L, and the computational results obtained for all of the second-order sensitivities of the PERP model’s leakage response with respect to the benchmark’s parameters listed in

Table 2. Summary of imprecisely known parameters for the PERP benchmark.

Symbol	Parameter Name	Number of Parameters
${\mathsf{\sigma }}_{t,i}^g$	Multigroup microscopic total cross section for isotope $i$ and energy group $g$	180 $for i=1, \dots , 6; g=1, \dots , 30$
${\mathsf{\sigma }}_{s,l,i}^{g^{\prime }\to g}$	Multigroup microscopic scattering cross section for $l$-th-order Legendre expansion, for isotope $i$, from energy group $g^{\prime }$ into group $g$	21,600 $\begin{array}{l}for l=0, \dots , 3; i=1, \dots , 6;\\ g,g^{\prime }=1, \dots , 30\end{array}$
${\mathsf{\sigma }}_{f,i}^g$	Multigroup microscopic fission cross section $i$ and energy group $g$	60 $for i=1,2; g=1, \dots , 30$
${\mathsf{\nu }}_i^g$	Average number of neutrons per fission for isotope $i$ and energy group $g$	60 $for i=1,2; g=1, \dots , 30$
${\mathsf{\chi }}_i^g$	Fission spectrum for isotope $i$ and energy group $g$	60 $for i=1,2; g=1, \dots , 30$
$q_j$	Source parameters: ${\mathsf{\lambda }}_1,{\mathsf{\lambda }}_2;F_1^{SF},F_2^{SF};a_1,a_2;b_1,b_2;{\mathsf{\nu }}_1^{SF},{\mathsf{\nu }}_2^{SF}$	10
$N_{i,m}$	Isotopic number density for isotope $i$ and material $m$	6 $N_{1,1},N_{2,1},N_{3,1},N_{4,1},N_{5,2},N_{6,2}$
$TP$	Total number of parameters	21,976

were reported in [19,20,21,22,23,24]. The computation of these second-order sensitivities was performed in the order of priority indicated by the values of the first-order sensitivities. Hence, the highest priority was given to computing the total of $\mathrm{32,400}=180\times 180$ second-order sensitivities of the PERP model’s leakage response to the model’s group-averaged total microscopic cross sections. It was thus found [19] that 720 of these elements have relative sensitivities greater than 1.0, and many of the second-order sensitivities are much larger than the corresponding first-order ones. The largest second-order sensitivities involve the total cross sections of ²³⁹Pu and ¹H. The overall largest element is the unmixed second-order relative sensitivity $S^{(2)}\left({\mathsf{\sigma }}_{t,6}^{30},{\mathsf{\sigma }}_{t,6}^{30}\right)=429.6$, which occurs in the lowest-energy group for ¹H.

The next largest sensitivities of the PERP leakage response were found to be [19,20,21,22,23,24] the mixed second-order sensitivities involving the isotopic number density of ²³⁹Pu and the microscopic total, scattering, or fission cross sections for the 12^th or 30^th energy groups of ²³⁹Pu or ¹H, respectively. On the other hand, the numerical results obtained in [19,20,21,22,23,24] indicated that the second-order sensitivities of the PERP leakage response with respect to the remaining parameters (including scattering cross sections, fission cross sections, average number of neutrons per fission, and source parameters) are much smaller than the corresponding first-order ones, so that the effects of uncertainties in these parameters on the expected value, variance, and skewness of the response distribution were negligible by comparison to the corresponding effects stemming from uncertainties in the total cross sections.

Neglecting the second-order sensitivities would cause an erroneous reporting of the response’s expected value and also a very large non-conservative error by under-reporting of the response variance. For example, if the parameters were uncorrelated and had a uniform standard deviation of 10%, neglecting second-order (and higher) sensitivities would cause a non-conservative error by under-reporting of the response variance by a factor of 947%. If the cross sections were fully correlated, neglecting the second-order sensitivities would cause an error as large as 2000% in the expected value of the leakage response, and up to 6000% in the variance of the leakage response. In all cases, neglecting the second-order sensitivities would erroneously predict a Gaussian distribution in parameter space (for the PERP leakage response) centered around the computed value of the leakage response. In reality, the second-order sensitivities cause the leakage distribution in parameter space to be skewed towards positive values relative to the expected value, which, in turn, is significantly shifted to much larger positive values than the computed leakage value.

The above-mentioned results for the second-order sensitivities indicated that the third-order sensitivities of the PERP leakage response with respect to the total cross sections (which were the most important parameters affecting the PERP leakage response) needed to be computed in order to assess their impact on the response and compare this impact to the corresponding impact of the second-order sensitivities. Recall that the PERP leakage response admits 180 first-order sensitivities and 32,400 s-order sensitivities (of which 16,290 are distinct from one another) with respect to the total cross sections. It follows that there are 5,832,000 third-order sensitivities (of which 988,260 are distinct from each other) of the leakage response (just) with respect to the total cross sections. The results for the third-order sensitivities of the PERP leakage response and their effects on the response’s expected value, variance, and skewness were reported in [26,27]. These results indicated that many third-order sensitivities were considerably larger than the corresponding second-order ones, which underscored the need to compute the fourth-order sensitivities corresponding to the largest third-order ones. There are 1,049,760,000 fourth-order sensitivities (of which 45,212,895 are distinct from each other) of the leakage response with respect to the total cross sections. The general methodology for computing fourth-order sensitivities exactly and efficiently while overcoming the curse of dimensionality has been developed by Cacuci [28] and applied to the PERP benchmark. It was found [29,30] that the microscopic total cross sections of isotopes ¹H and ²³⁹Pu are the most important parameters affecting the PERP benchmark’s leakage response, since they are involved in all of the large values (i.e., greater than 1.0) of the first-, second-, third- and fourth-order unmixed relative sensitivities. In particular, the largest unmixed fourth-order sensitivity is $S^{(4)}\left({\mathsf{\sigma }}_{t,6}^{g=30},{\mathsf{\sigma }}_{t,6}^{g=30},{\mathsf{\sigma }}_{t,6}^{g=30},{\mathsf{\sigma }}_{t,6}^{g=30}\right)=2.720\times {10}^6$ with respect to ${\mathsf{\sigma }}_{t,6}^{30}$ for the 30^th energy group of isotope ¹H. The 30^th energy group comprises thermalized neutrons in the energy interval from 1.39 × 10⁻⁴ eV to 0.152 eV. This sensitivity is about 90 times larger than the corresponding largest third-order relative sensitivity, about 6350 times larger than the corresponding largest second-order sensitivity, and about 291,000 times larger than the corresponding largest first-order relative sensitivity. The largest mixed fourth-order relative sensitivity is $S^{(4)}\left({\mathsf{\sigma }}_{t,6}^{30},{\mathsf{\sigma }}_{t,6}^{30},{\mathsf{\sigma }}_{t,6}^{30},{\mathsf{\sigma }}_{t,5}^{30}\right)=2.279\times {10}^5$, which involves the 30^th energy group of the microscopic total cross sections for isotopes #6 (¹H) and 5 (¹²C). Thus, among the 180 microscopic total cross sections, the parameter ${\mathsf{\sigma }}_{t,6}^{g=30}$, i.e., the total cross sections of isotope #6 (¹H) in the lowest energy group (30^th), is the single most important parameter affecting the PERP benchmark’s leakage response, as it has the largest impact on the various response moments. The comparative impact of the sensitivities of various orders, from first order to fourth order, are summarized in Section 3.1.2 below.

3.1.2. Overview of the Impacts of the First-, Second-, Third-, and Fourth-Order Sensitivities on Uncertainties Induced in the PERP’s Leakage Response

The effects of the first-, second-, third-, and fourth-order sensitivities on the expected value of the PERP benchmark’s leakage response are presented in

Table 3. Comparison of expected values for various relative standard deviations (RSD) of the normally distributed and uncorrelated microscopic total cross sections.

Expected Value	RSD = 1%	RSD = 5%	RSD = 10%
$L\left({\mathsf{\alpha }}^0\right)$	1.765 × 106	1.765 × 106	1.765 × 106
${\left[E\left(L\right)\right]}_t^{(2,U,N)}$	4.598 × 104	1.149 × 106	4.598 × 106
${\left[E\left(L\right)\right]}_t^{(4,U,N)}$	6.026 × 103	3.766 × 106	6.026 × 107
${\left[E\left(L\right)\right]}_t^{(U,N)}$	1.817 × 106	6.681 × 106	6.662 × 107

, considering that the total cross sections are normally distributed (indicated by the superscript “N”) and uncorrelated (indicated by the superscript “U”). As shown in Table 3, the effects of the second-order and fourth-order sensitivities through ${\left[E\left(L\right)\right]}_t^{\left(2,U,N\right)}\cong 2.5\%\times {\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$ and ${\left[E\left(L\right)\right]}_t^{(4,U,N)}\cong 0.3\%\times {\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$ to the expected response value ${\left[E\left(L\right)\right]}_t^{(U,N)}$ are negligibly small when considering a small relative standard deviation of 1% for each of the uncorrelated microscopic total cross sections of the isotopes included in the PERP benchmark. Considering a moderate (5%) relative standard deviation for each of the microscopic total cross sections, the results presented in Table 3 show that ${\left[E\left(L\right)\right]}_t^{\left(2,U,N\right)}\cong 65\%\times L\left({\mathsf{\alpha }}^0\right)\cong 17\%\times {\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$ indicate that the contributions from the second-order sensitivities to the expected response amount to ca. 65% of the computed leakage value $L\left({\mathsf{\alpha }}^0\right)$, and contribute ca. 17% to the expected value ${\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$ of the leakage response. Furthermore, the fourth-order sensitivities contribute around 56% to the expected value ${\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$, since ${\left[E\left(L\right)\right]}_t^{\left(4,U,N\right)}\cong 213\%\times L\left({\mathsf{\alpha }}^0\right)\cong 56\%\times {\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$. Hence, if the computed value, $L\left({\mathsf{\alpha }}^0\right)$, is considered to be the actual expected value of the leakage response, neglecting the fourth-order sensitivities would produce an error of ca. 210% for 5% relative standard deviations for uncorrelated total cross sections.

The rightmost column in Table 3 presents the results when considering a relative standard deviation of 10% for each of the uncorrelated microscopic total cross sections. These results indicate that ${\left[E\left(L\right)\right]}_t^{\left(2,U,N\right)}\cong 260\%\times L\left({\mathsf{\alpha }}^0\right)\cong 7\%\times {\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$, which implies that contributions of the second-order term are about 2.6 times larger than the computed leakage value $L\left({\mathsf{\alpha }}^0\right)$, and contribute around 7% to the expected value ${\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$. For the fourth-order term, the results in Table 3 show that ${\left[E\left(L\right)\right]}_t^{\left(4,U,N\right)}\cong 3414\%\times L\left({\mathsf{\alpha }}^0\right)\cong 90\%\times {\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$, indicating that the contributions from the fourth-order sensitivities to the expected response are around 34 times larger than the computed leakage value $L\left({\mathsf{\alpha }}^0\right)$ and contribute about 90% to the expected value ${\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$. Thus, for 10% relative standard deviations for uncorrelated total cross sections, the computed value $L\left({\mathsf{\alpha }}^0\right)$ would be ca. 3400% in error by comparison to the actual expected value of the leakage if the fourth-order sensitivities were neglected.

In summary, the impact of the fourth-order sensitivities on the expected value of the leakage response varies with the value of the standard deviation of the uncorrelated microscopic total cross sections. Generally, the larger the standard deviations of the microscopic total cross sections, the higher the impact of the fourth-order sensitivities will be on the expected value. For a small relative standard deviation of 1% for the parameters under consideration, the impact of the fourth-order sensitivities on the expected response value ${\left[E\left(L\right)\right]}_t^{(U,N)}$ is smaller than the impact of the lower-order sensitivities. However, for a moderate relative standard deviation of 5%, the contributions from the fourth-order sensitivities are around 56% of the expected value. When the relative standard deviation is increased to 10%, the contributions from the fourth-order sensitivities to the expected value increase to nearly 90%. Notably, for the “RSD = 10%” case, neglecting the fourth-order sensitivities would cause a large error (ca. 3400%) if the computed value $L\left({\mathsf{\alpha }}^0\right)$ were considered to be the actual expected value of the leakage response.

The effects of the first-, second-, third-, and fourth-order sensitivities on the variance of the leakage response are illustrated by the results presented in

Table 4. Comparison of variances for various relative standard deviations (RSD) of the normally distributed and uncorrelated microscopic total cross sections.

Variances	RSD = 1%	RSD = 5%	RSD = 10%
${\left[\mathrm{var} (L)\right]}_t^{\left(1,U,N\right)}$	3.419 × 1010	8.549 × 1011	3.419 × 1012
${\left[\mathrm{var} (L)\right]}_t^{\left(2,U,N\right)}$	2.879 × 109	1.799 × 1012	2.879 × 1013
${\left[\mathrm{var} (L)\right]}_t^{\left(3,U,N\right)}$	9.841 × 109	2.338 × 1013	1.236 × 1015
${\left[\mathrm{var} (L)\right]}_t^{\left(4,U,N\right)}$	1.825 × 109	2.852 × 1013	1.825 × 1015
${\left[\mathrm{var} (L)\right]}_t^{\left(U,N\right)}$	4.874 × 1010	5.456 × 1013	3.093 × 1015

. These effects are obtained when considering parameter relative standard deviations of 1%, 5%, and 10%, respectively, and considering that the total cross sections are normally distributed and uncorrelated. For uniform relative standard deviations of 1% for the uncorrelated microscopic total cross sections, the results presented in Table 4 indicate that ${\left[\mathrm{var} (L)\right]}_t^{(1,U,N)}\cong 70\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)},$ ${\left[\mathrm{var} (L)\right]}_t^{(2,U,N)}\cong 6\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$, ${\left[\mathrm{var} (L)\right]}_t^{(3,U,N)}\cong 20\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$, and ${\left[\mathrm{var} (L)\right]}_t^{(4,U,N)}\cong 4\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$. These results indicate that, for very small relative standard deviations (e.g., 1%), the contributions from the first-order sensitivities to the response variance are significantly larger (ca. 70%) than those from higher order sensitivities.

For uniform relative standard deviations of 5% for the uncorrelated microscopic total cross sections, the results presented in Table 4 indicate that $S{D}^{\left(4\right)}>S{D}^{\left(3\right)}\gg S{D}^{\left(2\right)}>S{D}^{\left(1\right)}$. Thus, the contributions from the third- and fourth-order sensitivities to the response variance are remarkably larger than those from the first- and second-order ones: ${\left[\mathrm{var} (L)\right]}_t^{(1,U,N)}\cong 2\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)},$ ${\left[\mathrm{var} (L)\right]}_t^{(2,U,N)}\cong 3\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$, ${\left[\mathrm{var} (L)\right]}_t^{(3,U,N)}\cong 43\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$, and ${\left[\mathrm{var} (L)\right]}_t^{(4,U,N)}\cong 52\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$. Hence, neglecting the fourth-order sensitivities would cause a significant error in quantifying the standard deviation of the leakage response for the PERP benchmark.

The results presented in the last column of Table 4 indicate that, for relative standard deviations of 10% for the uncorrelated microscopic total cross sections, ${\left[\mathrm{var} (L)\right]}_t^{(1,U,N)}\cong 0.1\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)},$ ${\left[\mathrm{var} (L)\right]}_t^{(2,U,N)}\cong 0.9\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$, ${\left[\mathrm{var} (L)\right]}_t^{(3,U,N)}\cong 40\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$, and ${\left[\mathrm{var} (L)\right]}_t^{(4,U,N)}\cong 59\%\times {\left[\mathrm{var} (L)\right]}_t^{(U,N)}$. Thus, the contributions from the third- and fourth-order sensitivities amount to ca. 99% of the total contribution to the response variance, while the contributions stemming from the first- and second-order sensitivities are negligibly small by comparison. The diverging trend displayed by the standard deviations of the leakage response as the order of sensitivity increases, namely, $S{D}^{\left(4\right)}>S{D}^{\left(3\right)}\gg S{D}^{\left(2\right)}>S{D}^{\left(1\right)}$, for the “10%” case is similar to the “5% case” above, but with much larger amplitudes.

The effects of the first-, second-, third-, and fourth-order sensitivities on the standard deviations of the leakage response when considering 10% relative standard deviations for the uncorrelated microscopic total cross sections are illustrated in Figure 2, which presents plots for $L\left({\mathsf{\alpha }}^0\right)\pm S{D}^{\left(1\right)}$, ${\left[E\left(L\right)\right]}_t^{(U,N)}\pm S{D}^{\left(1\right)}$, ${\left[E\left(L\right)\right]}_t^{(U,N)}\pm S{D}^{\left(2\right)}$, ${\left[E\left(L\right)\right]}_t^{(U,N)}\pm S{D}^{\left(3\right)}$, and ${\left[E\left(L\right)\right]}_t^{(U,N)}\pm S{D}^{\left(4\right)}$. Very importantly, the negative value of $L\left({\mathsf{\alpha }}^0\right)-S{D}^{\left(1\right)}$ shown in Figure 2 indicates that considering the computed value $L\left({\mathsf{\alpha }}^0\right)$ to be the expected value of the leakage response would produce unphysical results for 10% relative standard deviations for the microscopic total cross sections.

In summary, the impact of the sensitivities of various orders on the standard deviation of the leakage response is as follows: (i) for small (1%) relative standard deviations for the microscopic total cross sections, the uncertainty of the leakage response arising solely from the first-order sensitivities are significantly larger than the uncertainties arising solely from the second-, third-, and fourth-order sensitivities, respectively, but following an oscillating pattern: $S{D}^{\left(1\right)}\gg S{D}^{\left(3\right)}>S{D}^{\left(2\right)}>S{D}^{\left(4\right)}$; (ii) when considering moderate (5%) and larger (10%) relative standard deviations for the microscopic total cross sections, the standard deviation of the leakage response increases substantially as successively higher-order sensitivities are considered, leading to the following sequence of inequalities, $S{D}^{\left(4\right)}>S{D}^{\left(3\right)}\gg S{D}^{\left(2\right)}>S{D}^{\left(1\right)}$, which indicates the need to verify the convergence/divergence of the Taylor series which underlies the expression of the response’s standard deviation.

3.1.3. Effects of the Fourth-Order Sensitivities on the Third-Order Response Moment and Skewness

The results obtained for the third-order response moment and skewness when considering relative standard deviations of 1%, 5%, and 10%, respectively, for the microscopic total cross sections (considered to be normally distributed and uncorrelated) are summarized in

Table 5. Comparison of the third-order response moment and skewness for various relative standard deviations (RSD) of the normally distributed and uncorrelated microscopic total cross sections.

Third-Order Moment and Skewness	RSD = 1%	RSD = 5%	RSD = 10%
${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(2,U,N\right)}$	6.663 × 1015	1.070 × 1019	4.982 × 1020
${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(3,U,N\right)}$	3.948 × 1015	6.169 × 1019	3.948 × 1021
${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(4,U,N\right)}$	1.973 × 1015	3.083 × 1019	1.973 × 1021
${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$	1.258 × 1016	1.032 × 1020	6.419 × 1021
${\left[{\mathsf{\gamma }}_1(L)\right]}_t^{(U,N)}$	1.169	0.256	0.037

.

Considering 1% relative standard deviations for the uncorrelated microscopic total cross sections, the results presented in Table 5 indicate that ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(2,U,N\right)}\cong 53\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$, ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(3,U,N\right)}\cong 31\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$, and ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(4,U,N\right)}\cong 16\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$. Thus, for small (1%) relative standard deviations, the contributions to the third-order response moment ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$ stemming from the second-order sensitivities are the largest (e.g., around 53% in this case), followed by the contributions stemming from the third-order sensitivities, while the contributions stemming from the fourth-order sensitivities are the smallest.

Considering 5% relative standard deviations for the uncorrelated microscopic total cross sections, the results in Table 5 show that ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(2,U,N\right)}\cong 10\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$, ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(3,U,N\right)}\cong 60\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$, and ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(4,U,N\right)}\cong 30\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$. In this case, the contributions from the third-order sensitivities are the largest (e.g., around 60%), followed by the contributions from the fourth-order sensitivities, which contribute about 30%; the smallest contributions stem from the second-order sensitivities.

Considering 10% relative standard deviations for the uncorrelated microscopic total cross sections, the results in Table 5 indicate that ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(2,U,N\right)}\cong 8\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$, ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(3,U,N\right)}\cong 61\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$, and ${\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(4,U,N\right)}\cong 31\%\times {\left[{\mathsf{\mu }}_3 (L)\right]}_t^{\left(U,N\right)}$. These results display the same trends as the results for the “RSD = 5%” case, but the magnitudes of the respective contributions are significantly larger by comparison to the corresponding results for the “RSD = 5%” case.

For the response skewness, ${\left[{\mathsf{\gamma }}_1(L)\right]}_t^{(U,N)}$, the results shown in in the last row of Table 5 indicate that all the second-, third-, and fourth-order sensitivities produce a positive response skewness, which causes the leakage response distribution to be skewed towards the positive direction from its expected value ${\left[E\left(L\right)\right]}_t^{(U,N)}$. The results shown in the last row of Table 5 also indicate that as the relative standard deviation of the uncorrelated microscopic total cross sections increases from 1% to 10%, the value of the skewness decreases, thus causing the leakage response distribution to become increasingly more symmetrical about the mean value ${\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$. Neglecting the fourth-order sensitivities would cause a significant error in the skewness. For example, for the case “RSD = 5%”, if the fourth-order sensitivities were neglected, the contributions from the first-, second-, and third-order sensitivities to the skewness would have the value ${\left[{\mathsf{\gamma }}_1(L)\right]}_t^{(U,N)}=0.109$, which would be 135% smaller than the more accurate result obtained by including the contributions of the fourth-order sensitivities. The fourth-order sensitivities produce a positive response skewness, causing the leakage response distribution to be skewed towards the positive direction from its expected value. The impact of the fourth-order sensitivities on the skewness of the leakage response also changes with the value of the standard deviation of the microscopic total cross sections: larger standard deviations (for the parameters) tend to decrease the value of the skewness, causing the leakage response distribution to become more symmetrical about the mean value ${\left[E\left(L\right)\right]}_t^{\left(U,N\right)}$.

A “first-order sensitivity and uncertainty quantification” will always produce an erroneous third moment (and, hence, skewness) of the predicted response distribution, unless the unknown response distribution happens to be symmetrical. At least second-order sensitivities must be used in order to estimate the third-order moment (and, hence, the skewness) of the response distribution. With pronounced skewness, standard statistical inference procedures such as constructing a confidence interval for the mean (expectation) of a computed/predicted model response will be not only incorrect, in the sense that the true coverage level will differ from the nominal (e.g., 95%) level, but the error probabilities will be unequal on each side of the predicted mean. Thus, the truncation of Taylor expansion of the response (as a function of parameters) depends both on the magnitudes of the response sensitivities to parameters and the parameter uncertainties involved: if the uncertainties are small, then a fourth-order expansion suffices, in most cases, for obtaining relatively accurate results. In any case, the truncation error of a convergent Taylor series can be quantified a priori. If the parameter uncertainties are large, the Taylor series may diverge, so one would need to reduce such uncertainties, e.g., by performing additional measurements of the respective parameters. Of course, if the parameter uncertainties are large, all statistical methods are doomed to produce unreliable results for large-scale, realistic problems involving many uncertain parameters.

Obtaining the exact and complete set of first-order sensitivities of responses to model parameters is therefore of paramount importance for any analysis of a computational model. The second-order sensitivities contribute the leading correction terms to the response’s expected value, causing it to differ from the response’s computed value. The second-order sensitivities also contribute to the response variances and covariances. If the parameters follow a normal (Gaussian) multivariate distribution, the second-order sensitivities contribute the leading terms to the response’s third-order moment. In particular, the skewness of a single response, $R$, is customarily denoted as ${\mathsf{\gamma }}_1\left(R\right)$, and is defined as follows: ${\mathsf{\gamma }}_1\left(R\right)={\mathsf{\mu }}_3\left(R\right)/{\left[\mathrm{var}\left(R\right)\right]}^{\raisebox{1ex}{$3$}\!\left/ \!\raisebox{-1ex}{$2$}\right.},$ where ${\mathsf{\mu }}_3\left(R\right)$ denotes the third central moment of the response distribution. Thus, neglecting the second-order response sensitivities to normally distributed parameters would nullify the third-order response correlations and hence would nullify the skewness of a response. Hence, a “first-order sensitivity and uncertainty quantification” will always produce a zero-valued skewness of the predicted response distribution, which would be erroneous unless the unknown response distribution happens to be symmetrical. At least second-order sensitivities must be used in order to estimate the third-order moment and, hence, the skewness of the response distribution. The skewness provides a quantitative measure of the asymmetries in the respective distribution, indicating the direction and relative magnitude of a distribution’s deviation from the normal distribution. With pronounced skewness, standard statistical inference procedures such as constructing a confidence interval for the mean (expectation) of a computed/predicted model response will be incorrect, in the sense that the true coverage level will differ from the nominal (e.g., 95%) level. Furthermore, the error probabilities will be unequal on each side of the predicted mean.

3.2. Overview of the n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward and Adjoint Linear Systems (n^th-CASAM-L)

The n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward and Adjoint Linear Systems (n^th-CASAM-L), which was originally presented by Cacuci [36], is briefly summarized in this subsection. As was mentioned in Section 3.1, the mathematical/computational model of a physical system comprises model parameters which are imprecisely known. In addition to the TP model parameters $\mathsf{\alpha }\triangleq {\left({\mathsf{\alpha }}_1,\dots ,{\mathsf{\alpha }}_{TP}\right)}^{†}$ described in Section 3.1, the mathematical model of a physical system is considered to comprise $TI$ independent variables which will be denoted as $x_i, i=1, \dots , TI$, and are considered to be the components of a $TI$-dimensional column vector denoted as $x\triangleq {\left(x_1,\dots ,x_{TI}\right)}^{†}\in {ℝ}^{TI}$, where the sub/superscript “$TI$” denotes the “Total (number of) Independent variables” and where the components $x_i, i=1, \dots , TI$, can be considered, without loss of generality, to be real quantities. The vector $x\in {ℝ}^{TI}$ of independent variables is considered to be defined on a phase-space domain, denoted as $\mathsf{\Omega }\left(\mathsf{\alpha }\right)$, $\mathsf{\Omega }\left(\mathsf{\alpha }\right)\triangleq \left\{-\infty \le {\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)\le x_i\le {\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)\le \infty ; i=1, \dots , TI\right\}$, the boundaries of which may depend on (some of) the model parameters $\mathsf{\alpha }$. The lower boundary point of an independent variable is denoted as ${\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)$ (e.g., the inner radius of a sphere or cylinder, the lower range of an energy variable, the initial time value, etc.), while the corresponding upper boundary point is denoted as ${\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)$ (e.g., the outer radius of a sphere or cylinder, the upper range of an energy variable, the final time value, etc.). The boundary of $\mathsf{\Omega }\left(\mathsf{\alpha }\right)$ is denoted as $\partial \mathsf{\Omega }\left(\mathsf{\alpha }\right)\triangleq \left\{{\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)\cup {\mathsf{\omega }}_i\left(\mathsf{\alpha }\right), i=1,\dots ,TI\right\}$; it comprises the set of all of the endpoints ${\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right), {\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)$, $i=1, \dots , TI$.

A linear physical system is generally modeled by a system of coupled linear operator equations which can be generally represented in block-matrix form, as follows:

$$L\left(x;\mathsf{\alpha }\right)\mathsf{\phi }\left(x\right)=q_{\mathsf{\phi }}\left(x;\mathsf{\alpha }\right), x\in \mathsf{\Omega }\left(\mathsf{\alpha }\right)$$

(27)

where $\mathsf{\phi }\left(x\right)\triangleq {\left[{\mathsf{\phi }}_1\left(x\right),\dots ,{\mathsf{\phi }}_{TD}\left(x\right)\right]}^{†}$ is a $TD$-dimensional column vector of dependent variables and where the sub/superscript “$TD$” denotes the “Total (number of) Dependent variables.” The functions ${\mathsf{\phi }}_i\left(x\right), i=1,\dots ,TD$, denote the system’s “dependent variables” (also called “state functions”). The components $L_{ij}\left(x;\mathsf{\alpha }\right)$, $i,j=1,\dots ,TD$, of the $TD\times TD$-dimensional matrix $L\left(x;\mathsf{\alpha }\right) \triangleq \left[L_{ij}\left(x;\mathsf{\alpha }\right) \right]$ are operators that act linearly on the dependent variables ${\mathsf{\phi }}_j\left(\mathbf{x}\right)$ and also depend on the uncertain model parameters $\mathsf{\alpha }$. The components of the $TD$-dimensional column vector $q_{\mathsf{\phi }}\left(x;\mathsf{\alpha }\right) \triangleq {\left[q_{\mathsf{\phi },1}\left(x;\mathsf{\alpha }\right),\dots ,q_{\mathsf{\phi },TD}\left(x;\mathsf{\alpha }\right)\right]}^{†}$ represent inhomogeneous source terms which are considered to depend (nonlinearly) on the model parameters. Since the sources on the right side of Equation (27) may comprise distributions, all equalities that involve differential equations are considered to hold in the weak (i.e., “distributional”) sense.

Since the complete mathematical model is considered to be linear in $\mathsf{\phi }\left(x\right)$, the boundary and/or initial conditions needed to define the domain of $L\left(x;\mathsf{\alpha }\right)$; when $L\left(x;\mathsf{\alpha }\right)$ contains differential operators, it must also be linear in $\mathsf{\phi }\left(x\right)$ and can be represented as follows:

$$B_{\mathsf{\phi }}\left(x;\mathsf{\alpha }\right) \mathsf{\phi }\left(x\right)\mathbf{-}{c}_{\mathsf{\phi }}\left(x;\mathsf{\alpha }\right)=0, x\in \partial {\mathsf{\Omega }}_x\left(\mathsf{\alpha }\right).$$

(28)

In (28), the components $B_{ij}\left(x;\mathsf{\alpha }\right); i=1,\dots ,N_B; j=1,\dots ,TD$, of the $N_B\times TD$-dimensional matrix operator $B_{\mathsf{\phi }}\left(x;\mathsf{\alpha }\right)$ act linearly on $\mathsf{\phi }\left(x\right)$ and nonlinearly on $\mathsf{\alpha }$. The total number of boundary and initial conditions is denoted as $N_B$. The operator $c_{\mathsf{\phi }}\left(x;\mathsf{\alpha }\right)\triangleq {\left[c_{\mathsf{\phi },1}\left(x;\mathsf{\alpha }\right),\dots ,c_{\mathsf{\phi },N_B}\left(x;\mathsf{\alpha }\right)\right]}^{†}$ is a $N_B$-dimensional vector comprising inhomogeneous boundary source terms that act nonlinearly on $\mathsf{\alpha }$. The subscript “$\mathsf{\phi }$” in Equation (28) indicates boundary conditions associated with the forward state function $\mathsf{\phi }\left(x\right)$.

Physical problems modeled by linear systems and/or operators are naturally defined in Hilbert spaces. The dependent variables ${\mathsf{\phi }}_i\left(x\right), i=1,\dots ,TD$, for the physical system represented by Equations (27) and (28) are considered to be square-integrable functions of the independent variables, belonging to a Hilbert space which is denoted as ${\mathrm{H}}_0$ (where the subscript “zero” denotes “zeroth-level“ or “original”). The Hilbert space ${\mathrm{H}}_0$ is considered to be endowed with the following inner product, denoted as ${\langle \mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right)\rangle }_0$, between two elements $\mathsf{\phi }\left(x\right)\in {\mathrm{H}}_0$ and $\mathsf{\psi }\left(x\right)\in {\mathrm{H}}_0$:

$$\begin{array}{l}{\langle \mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right)\rangle }_0\triangleq {\displaystyle \underset{{\mathsf{\Omega }}_x}{\int }\mathsf{\phi }\left(x\right)·\mathsf{\psi }\left(x\right)d}x\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)}{\int }}\mathsf{\phi }\left(x\right)·\mathsf{\psi }\left(x\right) dx}} \\ ={\displaystyle \sum _{i=1}^{TD}{\displaystyle \underset{{\mathsf{\lambda }}_1\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_1\left(\mathsf{\alpha }\right)}{\int }}\dots {\displaystyle \underset{{\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)}{\int }}\dots {\displaystyle \underset{{\mathsf{\lambda }}_{TI}\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_{TI}\left(\mathsf{\alpha }\right)}{\int }}{\mathsf{\phi }}_i\left(\mathbf{x}\right){\mathsf{\psi }}_i\left(\mathbf{x}\right) d{x}_1 d{x}_2\dots d{x}_i\dots d{x}_{TI}}}}}.\end{array}$$

(29)

The product notation ${\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)}{\int }}[ ] d{x}_i}}$ in Equation (29) denotes the respective multiple integrals. The linear operator $L\left(x;\mathsf{\alpha }\right)$ admits an adjoint operator, which is denoted as $L^{*}\left(x;\mathsf{\alpha }\right)$ and which is defined through the following relation for a vector $\mathsf{\psi }\left(x\right)\in {\mathrm{H}}_0$:

$${\langle \mathsf{\psi }\left(x\right), L\left(x;\mathsf{\alpha }\right) \mathsf{\phi }\left(x\right)\rangle }_0={\langle L^{*}\left(x;\mathsf{\alpha }\right) \mathsf{\psi }\left(x\right), \mathsf{\phi }\left(x\right)\rangle }_0$$

(30)

In (30), the formal adjoint operator $L^{*}\left(x;\mathsf{\alpha }\right)$ is the $TD\times TD$ matrix comprising elements $L_{ji}^{*}\left(x;\mathsf{\alpha }\right)$ obtained by transposing the formal adjoints of the forward operators $L_{ij}\left(x;\mathsf{\alpha }\right)$, i.e.,

$$L^{*}\left(x;\mathsf{\alpha }\right) \triangleq \left[L_{ji}^{*}\left(x;\mathsf{\alpha }\right) \right], i,j=1,\dots ,TD,$$

(31)

Hence, the system adjoint to the linear system represented by (27) and (28) can generally be represented as follows:

$${\mathbf{L}}^{*}\left(x;\mathsf{\alpha }\right) \mathsf{\psi }\left(x\right)=q_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right), x\in {\mathsf{\Omega }}_x\left(\mathsf{\alpha }\right) ,$$

(32)

$$B_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right) \mathsf{\psi }\left(x\right) -c_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right) =0, x\in \partial {\mathsf{\Omega }}_x\left(\mathsf{\alpha }\right) .$$

(33)

When $L\left(x;\mathsf{\alpha }\right)$ comprises differential operators, the operations (e.g., integration by parts) that implement the transition from the left side to the right side of Equation (30) give rise to boundary terms which are collectively called the “bilinear concomitant.” The domain of $L^{*}\left(x;\mathsf{\alpha }\right)$ is determined by selecting adjoint boundary and/or initial conditions so as to ensure that the bilinear concomitant vanishes when the selected adjoint boundary conditions are implemented together with the forward boundary conditions given in Equation (28). The adjoint boundary conditions thus selected are represented in operator form by Equation (33), where the subscript “$\mathsf{\psi }$” indicates adjoint boundary and/or initial conditions associated with the adjoint state function $\mathsf{\psi }\left(x\right)$. In most practical situations, the Hilbert space ${\mathrm{H}}_0$ is self-dual. Solving Equations (27) and (28) at the nominal parameter values ${\mathsf{\alpha }}^0$ yields the nominal value ${\mathsf{\phi }}^0\left(x\right)$ of the forward state function, while solving Equations (32) and (33) at ${\mathsf{\alpha }}^0$ yields the nominal value ${\mathsf{\psi }}^0\left(x\right)$ of the adjoint state function.

The relationship shown in Equation (30), which is the basis for defining the adjoint operator, also provides the following fundamental “reciprocity-like” relation between the sources of the forward and the adjoint equations, cf. Equations (27) and (32), respectively:

$${\langle \mathsf{\psi }\left(x\right), q_{\mathsf{\phi }}\left(x;\mathsf{\alpha }\right)\rangle }_0={\langle q_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right), \mathsf{\phi }\left(x\right)\rangle }_0$$

(34)

The functional on the right side of Equation (34) represents a “detector response”, i.e., a reaction rate between the particles and the medium represented by $q_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right)$, which is equivalent to the “number of counts” of particles incident on a detector of particles that “measures” the particle flux $\mathsf{\phi }\left(x\right)$. In view of the relation provided in Equation (34), the source term $q_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right) \triangleq {\left[q_{\mathsf{\psi },1}\left(x;\mathsf{\alpha }\right),\dots ,q_{\mathsf{\psi },TD}\left(x;\mathsf{\alpha }\right)\right]}^{†}$ in the adjoint Equation (32) is usually associated with the “result of interest” to be measured and/or computed, which is customarily called the system’s “response.” In particular, if $q_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right)=\mathsf{\delta }\left(x-x_d\right)$, then ${\langle q_{\mathsf{\psi }}\left(x;\mathsf{\alpha }\right), \mathsf{\phi }\left(x\right)\rangle }_0=\mathsf{\phi }\left(x_d\right)$, which means that, in such a case, the right side of (34) provides the value of the dependent variable (particle flux, temperature, velocity, etc.) at the point in the phase space where the respective measurement is performed.

Since the linear operators that underly linear systems admit adjoint operators, important model responses of linear systems can be functions of both the forward and the adjoint state functions. Examples of such responses are the Schwinger [37] and Roussopoulos [38] functionals, as well as the Raleigh quotient [39] for computing eigenvalues of linear systems. Such responses cannot be encountered in nonlinear problems since nonlinear operators do not admit adjoint operators. In this sense, linear systems differ fundamentally from nonlinear systems and cannot always be considered to be particular cases of nonlinear systems but need to be treated comprehensively in their own right. Thus, the fundamental form of a response computed using a linear model is an operator which acts nonlinearly on the model’s forward and adjoint state functions, as well as on the imprecisely known model parameters, directly and indirectly through the state functions, having the following generic form:

$$R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)}{\int }}S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]d{x}_i}},$$

(35)

where $S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ denotes a suitably Gateaux (G)-differential function of the indicated arguments. In general, $S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ is nonlinear in $\mathsf{\phi }\left(x\right)$, $\mathsf{\psi }\left(x\right)$, and $\mathsf{\alpha }$, and the components of $\mathsf{\alpha }$ are considered to also include parameters that may specifically appear only in the definition of the response under consideration (but which might not appear in the definition of the model). Thus, the (physical) “system” is defined and understood to comprise both the system’s computational model and the system’s response. Due to the response’s implicit dependence on the model parameters through the forward and/or adjoint state functions, the response of a linear system is a nonlinear function of the model’s parameters, except perhaps in particularly trivial situations. The nominal value of the response, $R\left[{\mathsf{\phi }}^0\left(x\right),{\mathsf{\psi }}^0\left(x\right);{\mathsf{\alpha }}^0\right]$, is determined by using the nominal parameter values ${\mathsf{\alpha }}^0$, the nominal value ${\mathsf{\phi }}^0\left(x\right)$ of the forward state function, and the nominal value ${\mathsf{\psi }}^0\left(x\right)$ of the adjoint state function.

Since the model parameters ${\mathsf{\alpha }}_i$ are imprecisely known quantities, their actual values (which are not known in practice) will differ from their nominal values (which are known in practice) by quantities denoted as $\mathsf{\delta }{\mathsf{\alpha }}_i\triangleq {\mathsf{\alpha }}_i-{\mathsf{\alpha }}_i^0, i=1,\dots ,TP.$ The variations $\mathsf{\delta }\mathsf{\alpha }\triangleq \left(\mathsf{\delta }{\mathsf{\alpha }}_1,\dots ,\mathsf{\delta }{\mathsf{\alpha }}_{TP}\right)$ will cause corresponding variations $\mathsf{\delta }\mathsf{\phi }\triangleq \left(\mathsf{\delta }{\mathsf{\phi }}_1, \dots , \mathsf{\delta }{\mathsf{\phi }}_{TD}\right),$$\mathsf{\delta }{\mathsf{\phi }}_i\triangleq {\mathsf{\phi }}_i-{\mathsf{\phi }}_i^0,$ $\mathsf{\delta }\mathsf{\psi }\triangleq \left(\mathsf{\delta }{\mathsf{\psi }}_1,\dots ,\mathsf{\delta }{\mathsf{\psi }}_{TD}\right)$,$\mathsf{\delta }{\mathsf{\psi }}_i\triangleq {\mathsf{\psi }}_i-{\mathsf{\psi }}_i^0,$$i=1,\dots ,TD$, in the forward and adjoint state functions, respectively. In turn, all of these variations will cause a response variation $R\left({\mathsf{\phi }}^0+\mathsf{\delta }\mathsf{\phi };{\mathsf{\psi }}^0+\mathsf{\delta }\mathsf{\psi };{\mathsf{\alpha }}^0+\mathsf{\delta }\mathsf{\alpha }\right)$ around the nominal value $R\left[{\mathsf{\phi }}^0\left(x\right),{\mathsf{\psi }}^0\left(x\right);{\mathsf{\alpha }}^0\right]$. The first-order sensitivities of the response with respect to the model parameters are contained within the first-order Gateaux (G)-variation, denoted as $\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi },\mathsf{\delta }\mathsf{\alpha }\right)$, of the response $R\left(e\right)$ for arbitrary variations $\mathsf{\delta }\mathsf{\phi }\left(x\right),\mathsf{\delta }\mathsf{\psi }\left(x\right),\mathsf{\delta }\mathsf{\alpha }$ in the model parameters and state functions, in a neighborhood $\left[{\mathsf{\phi }}^0\left(x\right)+\mathsf{\epsilon }\mathsf{\delta }\mathsf{\phi }\left(x\right),{\mathsf{\psi }}^0\left(x\right)+\mathsf{\epsilon }\mathsf{\delta }\mathsf{\psi }\left(x\right);{\mathsf{\alpha }}^0+\mathsf{\epsilon }\mathsf{\delta }\mathsf{\alpha }\right]$ around $\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0\right)$. By definition, the first-order Gateaux (G)-variation $\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi },\mathsf{\delta }\mathsf{\alpha }\right)$ is obtained as follows:

$$\begin{array}{l}\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi },\mathsf{\delta }\mathsf{\alpha }\right)\\ \triangleq {\left\{\frac{d}{d\mathsf{\epsilon }}R\left[{\mathsf{\phi }}^0\left(x\right)+\mathsf{\epsilon }\mathsf{\delta }\mathsf{\phi }\left(x\right),{\mathsf{\psi }}^0\left(x\right)+\mathsf{\epsilon }\mathsf{\delta }\mathsf{\psi }\left(x\right);{\mathsf{\alpha }}^0+\mathsf{\epsilon }\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{\mathsf{\epsilon }=0}\\ \triangleq {\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi }\right)\right\}}_{dir}+{\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{ind},\end{array}$$

(36)

where the “direct-effect” term ${\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{dir}$ depends only on the parameter variation $\mathsf{\delta }\mathsf{\alpha }$ and is defined as follows:

$$\begin{array}{l}{\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{dir}\triangleq {\left\{\frac{\partial }{\partial \mathsf{\alpha }}{\displaystyle \prod _{k=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_k\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_k\left(\mathsf{\alpha }\right)}{\int }}d{x}_{k}S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]}}\right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha }\\ ={\displaystyle \prod _{k=1}^{TI}{\left\{{\displaystyle \underset{{\mathsf{\kappa }}_k\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_k\left(\mathsf{\alpha }\right)}{\int }}d{x}_k\frac{\partial S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]}{\partial \mathsf{\alpha }}\mathsf{\delta }\mathsf{\alpha }}\right\}}_{{\mathsf{\alpha }}^0}}\\ +{\displaystyle \sum _{k=1}^{TI}{\displaystyle \prod _{m=1,k\ne j}^{TI}{\left\{{\displaystyle \underset{{\mathsf{\lambda }}_m\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_m\left(\mathsf{\alpha }\right)}{\int }}d{x}_{m}S \left[\mathsf{\phi }\left(\dots ,{\mathsf{\omega }}_k,\dots ,x_{N_x}\right),\mathsf{\psi }\left(\dots ,{\mathsf{\omega }}_k,\dots \right);\mathsf{\alpha }\right]\frac{\partial {\mathsf{\omega }}_k\left(\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}}\right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha }}}\\ -{\displaystyle \sum _{k=1}^{TI}{\displaystyle \prod _{m=1,k\ne j}^{TI}{\left\{{\displaystyle \underset{{\mathsf{\lambda }}_m\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_m\left(\mathsf{\alpha }\right)}{\int }}d{x}_m}S \left[\mathsf{\phi }\left(\dots ,{\mathsf{\lambda }}_k,\dots \right),\mathsf{\psi }\left(\dots ,{\mathsf{\lambda }}_k,\dots \right);\mathsf{\alpha }\right]\frac{\partial {\mathsf{\lambda }}_k\left(\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}\right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha }}},\end{array}$$

(37)

and where the “indirect effect” term ${\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi }\right)\right\}}_{ind}$ depends only on the variations $\mathsf{\delta }\mathsf{\phi }$ and $\mathsf{\delta }\mathsf{\psi }$ in the state functions, and is defined as follows:

$$\begin{array}{l}{\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi }\right)\right\}}_{ind} \\ \triangleq {\displaystyle \prod _{k=1}^{TI}{\left\{{\displaystyle \underset{{\mathsf{\lambda }}_k\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_k\left(\mathsf{\alpha }\right)}{\int }}d{x}_k\left[\frac{\partial S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]}{\partial \mathsf{\phi }}\mathsf{\delta }\mathsf{\phi }+\frac{\partial S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]}{\partial \mathsf{\psi }}\mathsf{\delta }\mathsf{\psi }\right]}\right\}}_{{\mathsf{\alpha }}^0}}.\end{array}$$

(38)

The notation ${\left\{\right\}}_{{\mathsf{\alpha }}^0}$ has been used in Equations (37) and (38) to indicate that the quantity within the brackets is to be evaluated at the nominal values of the parameters and state functions. While the “direct effect” term ${\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{dir}$ defined in Equation (37), which depends only on the parameter variations $\mathsf{\delta }\mathsf{\alpha }$, can be computed immediately, the “indirect effect” term ${\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi }\right)\right\}}_{ind}$ defined in Equation (38) can be computed only after having determined the values of the variations $\mathsf{\delta }\mathsf{\phi }\left(x\right)$ and $\mathsf{\delta }\mathsf{\psi }\left(x\right)$. The variations $\mathsf{\delta }\mathsf{\phi }\left(x\right)$ and $\mathsf{\delta }\mathsf{\psi }\left(x\right)$ are the solutions of the system of equations obtained by taking the G-differentials of Equations (27), (28), (32), and (33), which can be written in the following matrix-vector form:

$${\left\{V^{\left(1\right)}\left(\mathsf{\alpha }\right)\mathsf{\delta }{u}^{\left(1\right)}\left(x\right)\right\}}_{{\mathsf{\alpha }}^0}={\left\{q^{\left(1\right)}\left(u^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}, x\in {\mathsf{\Omega }}_x,$$

(39)

$${\left\{b_v^{\left(1\right)}\left(u^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }{u}^{\left(1\right)} ;\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right),$$

(40)

where

$$\begin{array}{l}{V}^{\left(1\right)}\left(\mathsf{\alpha }\right)\triangleq \left(\begin{array}{cc}L\left(\mathsf{\alpha }\right)& 0\\ 0& L^{*}\left(\mathsf{\alpha }\right)\end{array}\right); u^{\left(1\right)}\left(x\right)\triangleq \left(\begin{array}{c}\mathsf{\phi }\left(x\right)\\ \mathsf{\psi }\left(x\right)\end{array}\right); \mathsf{\delta }{u}^{\left(1\right)}\left(x\right)\triangleq \left(\begin{array}{c}\mathsf{\delta }\mathsf{\phi }\left(x\right)\\ \mathsf{\delta }\mathsf{\psi }\left(x\right)\end{array}\right); \\ q^{\left(1\right)}\left(u^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\triangleq \left(\begin{array}{c}{q}_1^{\left(1\right)}\left(\mathsf{\phi };\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\\ q_2^{\left(1\right)}\left(\mathsf{\psi };\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\end{array}\right); b_v^{\left(1\right)}\left(u^{\left(1\right)}; \mathsf{\delta }{u}^{\left(1\right)} ;\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\triangleq \left(\begin{array}{c}{b}_1^{\left(1\right)}\left(\mathsf{\phi };\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\\ b_2^{\left(1\right)}\left(\mathsf{\psi };\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\end{array}\right).\end{array}$$

(41)

$${\left\{q_1^{(1)}\left(\mathsf{\phi },\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right) \right\}}_{{\mathsf{\alpha }}^0}\triangleq {\left\{\frac{\partial \left[q_{\mathsf{\phi }}\left(\mathsf{\alpha }\right)-L\left(\mathsf{\alpha }\right)\mathsf{\phi }\right]}{\partial \mathsf{\alpha }} \right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha },$$

(42)

$${\left\{q_2^{(1)}\left(\mathsf{\psi },\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right) \right\}}_{{\mathsf{\alpha }}^0}\triangleq {\left\{\frac{\partial \left[q_{\mathsf{\psi }}\left(\mathsf{\alpha }\right)-L^{*}\left(\mathsf{\alpha }\right)\mathsf{\psi }\left(x\right)\right]}{\partial \mathsf{\alpha }} \right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha },$$

(43)

The matrices $\partial q_{\mathsf{\phi }}\left(\mathsf{\alpha }\right)/\partial \mathsf{\alpha }$ and $\partial \left[L\left(\mathsf{\alpha }\right)\mathsf{\phi }\right]/\partial \mathsf{\alpha }$, which appear on the right side of Equation (39), are defined as follows:

$$\frac{\partial q_{\mathsf{\phi }}\left(\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}\triangleq \left(\begin{array}{ccc}·& \dots & ·\\ ⋮& \frac{\partial q_{\mathsf{\phi },i}}{\partial {\mathsf{\alpha }}_j}& ⋮\\ ·& \cdots & ·\end{array}\right) , \frac{\partial \left[L\left(\mathsf{\alpha }\right)\mathsf{\phi }\right]}{\partial \mathsf{\alpha }}\triangleq \left(\begin{array}{ccc}·& \dots & ·\\ ⋮& \frac{\partial \left[{\displaystyle \sum _{k=1}^{TD}{L}_{i,k}\left(\mathsf{\alpha }\right){\mathsf{\phi }}_k}\right]}{\partial {\mathsf{\alpha }}_j}& ⋮\\ ·& \cdots & ·\end{array}\right) .$$

(44)

The system of equations comprising Equations (39) and (40) is called the “1^st-Level Variational Sensitivity System” (1^st-LVSS) and its solution, $\mathsf{\delta }{u}^{\left(1\right)}\left(x\right)$, is called the “1^st-Level variational sensitivity function”, as indicated by the superscript “(1)”. Since the source term, $q^{\left(1\right)}\left(u^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)$, is a function of $\mathsf{\delta }\mathsf{\alpha }$, it follows that the 1^st-LVSS would need to be solved $TP$ times in order to obtain the vector $\mathsf{\delta }{u}^{\left(1\right)}\left(x\right)$ which would correspond to each parameter variation $\mathsf{\delta }{\mathsf{\alpha }}_i$, $j_1=1,\dots ,TP$. The need for computing the functions $\mathsf{\delta }\mathsf{\phi }\left(x\right)$ and $\mathsf{\delta }\mathsf{\psi }\left(x\right)$ can be avoided by applying the First-Order Comprehensive Adjoint Sensitivity Analysis Methodology (1^st-CASAM-L), which enables the elimination of $\mathsf{\delta }\mathsf{\phi }\left(x\right)$ and $\mathsf{\delta }\mathsf{\psi }\left(x\right)$ from the expression of the indirect effect term ${\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi }\right)\right\}}_{ind}$ defined in Equation (38). This elimination is achieved by implementing the following sequence of steps:

• Introduce a Hilbert space, denoted as ${\mathrm{H}}_1$, comprising square-integrable function vector-valued elements of the form ${\mathsf{\eta }}^{\left(1\right)}\left(x\right)\triangleq {\left[{\mathsf{\eta }}_1^{\left(1\right)}\left(x\right),{\mathsf{\eta }}_2^{\left(1\right)}\left(x\right)\right]}^{†}$, ${\mathsf{\eta }}_i^{\left(1\right)}\left(x\right)\triangleq {\left[{\mathsf{\eta }}_{i,1}^{\left(1\right)}\left(x\right),\dots ,{\mathsf{\eta }}_{i,j}^{\left(1\right)}\left(x\right),\dots ,{\mathsf{\eta }}_{i,TD}^{\left(1\right)}\left(x\right)\right]}^{†}$, $i=1,2$. The inner product between two elements, ${\mathsf{\eta }}^{\left(1\right)}\left(x\right)\in {\mathrm{H}}_1$, ${\mathsf{\xi }}^{\left(1\right)}\left(x\right)\in {\mathrm{H}}_1$, is denoted as ${\langle {\mathsf{\eta }}^{\left(1\right)}\left(x\right),{\mathsf{\xi }}^{\left(1\right)}\left(x\right)\rangle }_1$ and is defined as follows:

  $${\langle {\mathsf{\eta }}^{\left(1\right)}\left(x\right),{\mathsf{\xi }}^{\left(1\right)}\left(x\right)\rangle }_1\triangleq {\displaystyle \sum _{i=1}^2{\langle {\mathsf{\eta }}_i^{\left(1\right)}\left(x\right),{\mathsf{\xi }}_i^{\left(1\right)}\left(x\right)\rangle }_0}$$

  (45)
• In the Hilbert ${\mathrm{H}}_1$, form the inner product of Equation (39) with a vector-valued function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}\in {\mathrm{H}}_1$ to obtain:

  $${\left\{{\langle a^{\left(1\right)}\left(x\right),V^{\left(1\right)}\left(\mathsf{\alpha }\right)\mathsf{\delta }{u}^{\left(1\right)}\left(x\right)\rangle }_1\right\}}_{{\mathsf{\alpha }}^0}={\left\{{\langle a^{\left(1\right)}\left(x\right),q^{\left(1\right)}\left(\mathsf{\phi };\mathsf{\psi };\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right) \rangle }_1\right\}}_{{\mathsf{\alpha }}^0}$$

  (46)
• Using the definition of the adjoint operator in the Hilbert space ${\mathrm{H}}_1$, recast the left side of Equation (46) as follows:

  $$\begin{array}{l}{\left\{{\langle a^{\left(1\right)}\left(x\right),V^{\left(1\right)}\left(\mathsf{\alpha }\right)\mathsf{\delta }{u}^{\left(1\right)}\left(x\right)\rangle }_1\right\}}_{{\mathsf{\alpha }}^0}={\left\{{\langle \mathsf{\delta }{u}^{\left(1\right)}\left(x\right),A^{\left(1\right)}\left(\mathsf{\alpha }\right)a^{\left(1\right)}\left(x\right) \rangle }_1\right\}}_{{\mathsf{\alpha }}^0}\\ +{\left\{P^{\left(1\right)}\left[\mathsf{\delta }{u}^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0},\end{array}$$

  (47)

  where ${\left\{P^{\left(1\right)}\left[\mathsf{\delta }{u}^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ denotes the bilinear concomitant defined on the phase-space boundary $x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right)$, and where $A^{\left(1\right)}\left(\mathsf{\alpha }\right)$ is the operator formally adjoint to $V^{\left(1\right)}\left(\mathsf{\alpha }\right)$, defined as follows:

  $$A^{\left(1\right)}\left(\mathsf{\alpha }\right)\triangleq {\left[V^{\left(1\right)}\left(\mathsf{\alpha }\right)\right]}^{*}=\left(\begin{array}{cc}{L}^{*}\left(\mathsf{\alpha }\right)& 0\\ 0& L\left(\mathsf{\alpha }\right)\end{array}\right)$$

  (48)
• Require the first term on right side of Equation (47) to represent the indirect effect term defined in Equation (38), to obtain the following relation:

  $$A^{\left(1\right)}\left(\mathsf{\alpha }\right)a^{\left(1\right)}\left(x\right) =s^{\left(1\right)}\left( u^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right),$$

  (49)

  where

  $$s^{\left(1\right)}\left[u^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]\triangleq \left(\begin{array}{c}{\left[s_1^{\left(1\right)}\left( u^{\left(1\right)};\mathsf{\alpha }\right)\right]}^{†}\\ {\left[s_2^{\left(1\right)}\left( u^{\left(1\right)};\mathsf{\alpha }\right)\right]}^{†}\end{array}\right)\triangleq \left(\begin{array}{c}{\left[\partial S\left( u^{\left(1\right)};\mathsf{\alpha }\right)/\partial \mathsf{\phi }\right]}^{†}\\ {\left[\partial S\left( u^{\left(1\right)};\mathsf{\alpha }\right)/\partial \mathsf{\psi }\right]}^{†}\end{array}\right) .$$

  (50)
• Implement the boundary conditions given in Equation (40) into Equation (47) and eliminate the remaining unknown boundary values of the functions $\mathsf{\delta }\mathsf{\phi }$ and $\mathsf{\delta }\mathsf{\psi }$ from the expression of the bilinear concomitant ${\left\{P^{\left(1\right)}\left[\mathsf{\delta }{u}^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ by selecting appropriate boundary conditions for the function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$, to ensure that Equation (49) is well-posed while being independent of unknown values of $\mathsf{\delta }\mathsf{\phi }$, $\mathsf{\delta }\mathsf{\psi }$, and $\mathsf{\delta }\mathsf{\alpha }$. The boundary conditions thus chosen for the function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$ can be represented in operator form as follows:

  $${\left\{b_A^{\left(1\right)}\left[ u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\right]\right\}}_{{\mathsf{\alpha }}^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right).$$

  (51)
• The selection of the boundary conditions for the adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$ represented by Equation (51) eliminates the appearance of the unknown values of $\mathsf{\delta }{u}^{\left(1\right)}\left(x\right)$ in the bilinear concomitant ${\left\{P^{\left(1\right)}\left[\mathsf{\delta }{u}^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ and reduces it to a residual quantity which will be denoted as ${\left\{{\widehat{P}}^{\left(1\right)}\left[ u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ and which contains boundary terms involving only known values of $\mathsf{\phi }\left(x\right)$, $\mathsf{\psi }\left(x\right)$, $a^{\left(1\right)}\left(x\right)$, $\mathsf{\alpha }$, and $\mathsf{\delta }\mathsf{\alpha }$. This residual quantity does not automatically vanish.
• The system of equations comprising Equation (49) together with the boundary conditions represented in Equation (51) constitute the First-Level Adjoint Sensitivity System (1^st-LASS). The solution $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$ of the 1^st-LASS is called the First-level adjoint sensitivity function. The 1^st-LASS is called “first-level” (as opposed to “first-order”) because it does not contain any differential or functional derivatives, but its solution, $a^{\left(1\right)}\left(x\right)$, is used below to compute the first-order sensitivities of the response with respect to the model parameters.
• It follows from (46) and (47) that the following relation holds:

  $$\begin{array}{l}{\left\{{\langle a^{\left(1\right)}\left(x\right),q^{\left(1\right)}\left( u^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right) \rangle }_1\right\}}_{{\mathsf{\alpha }}^0}={\left\{{\langle \mathsf{\delta }{u}^{\left(1\right)}\left(x\right),A^{\left(1\right)}\left(\mathsf{\alpha }\right)a^{\left(1\right)}\left(x\right) \rangle }_1\right\}}_{{\mathsf{\alpha }}^0}\\ +{\left\{{\widehat{P}}^{\left(1\right)}\left[ u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}.\end{array}$$

  (52)
• Recalling from (49) that the first term on the right side of (52) is the indirect effect term ${\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi }\right)\right\}}_{ind}$, it follows from (52) that the indirect effect term can be expressed in terms of the first-level adjoint sensitivity function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),{\mathbf{a}}_2^{\left(1\right)}\left(x\right)\right]}^{†}$ as follows:

  $$\begin{array}{l}{\left\{\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0,{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi }\right)\right\}}_{ind}={\left\{{\langle a^{\left(1\right)}\left(x\right),q^{\left(1\right)}\left( u^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right) \rangle }_1\right\}}_{{\mathsf{\alpha }}^0} \\ -{\left\{{\widehat{P}}^{\left(1\right)}\left[u^{\left(1\right)};a^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\equiv {\left\{\mathsf{\delta }R\left(u^{\left(1\right)};a^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{ind}.\end{array}$$

  (53)

As indicated by the identity relation in Equation (53), the variations $\mathsf{\delta }\mathsf{\phi }$ and $\mathsf{\delta }\mathsf{\psi }$ have been eliminated from the original expression of the indirect effect term, which now instead depends on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$. Very importantly, the 1^st-LASS is independent of parameter variations $\mathsf{\delta }\mathsf{\alpha }$. Hence, the 1^st-LASS needs to be solved only once (as opposed to the 1^st-LVSS or the 1^st-OFSS, which would need to be solved anew for each parameter variation) to determine the first-level adjoint sensitivity function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$. Subsequently, the indirect effect term ${\left\{\mathsf{\delta }R\left(u^{\left(1\right)};a^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{ind}$ is computed efficiently and exactly by simply performing the integrations over the adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$, as indicated by Equation (53).

Adding the expression of the direct effect term defined in Equation (37) to the expression of the indirect effect term obtained in Equation (53) yields the following expression for the total first-order G-differential ${\left\{\mathsf{\delta }R\left[u^{\left(1\right)}\left(x\right),\mathsf{\alpha };\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi },\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ of the response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ computed at the nominal values for the models’ parameter and state functions:

$$\begin{array}{l}{\left\{\mathsf{\delta }R\left[u^{\left(1\right)}\left(x\right),\mathsf{\alpha };\mathsf{\delta }\mathsf{\phi },\mathsf{\delta }\mathsf{\psi },\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\\ ={\left\{{\displaystyle \prod _{k=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_k\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_k\left(\mathsf{\alpha }\right)}{\int }}d{x}_k\frac{\partial S \left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]}{\partial \mathsf{\alpha }}\mathsf{\delta }\mathsf{\alpha }}}\right\}}_{{\mathsf{\alpha }}^0}\\ +{\displaystyle \sum _{k=1}^{TI}{\displaystyle \prod _{m=1,k\ne j}^{TI}{\left\{{\displaystyle \underset{{\mathsf{\lambda }}_m\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_m\left(\mathsf{\alpha }\right)}{\int }}d{x}_{m}S \left[\mathsf{\phi }\left(\dots ,{\mathsf{\omega }}_k,\dots ,x_{N_x}\right),\mathsf{\psi }\left(\dots ,{\mathsf{\omega }}_k,\dots \right);\mathsf{\alpha }\right]\frac{\partial {\mathsf{\omega }}_k\left(\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}}\right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha }}}\\ -{\displaystyle \sum _{k=1}^{TI}{\displaystyle \prod _{m=1,k\ne j}^{TI}{\left\{{\displaystyle \underset{{\mathsf{\lambda }}_m\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_m\left(\mathsf{\alpha }\right)}{\int }}d{x}_m}S \left[\mathsf{\phi }\left(\dots ,{\mathsf{\lambda }}_k,\dots \right),\mathsf{\psi }\left(\dots ,{\mathsf{\lambda }}_k,\dots \right);\mathsf{\alpha }\right]\frac{\partial {\mathsf{\lambda }}_k\left(\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}\right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha }}},\\ +{\left\{{\displaystyle \sum _{i=1}^2{\langle a_i^{\left(1\right)}\left(x\right),q_i^{\left(1\right)}\left(u^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\rangle }_0}\right\}}_{{\mathsf{\alpha }}^0}-{\left\{{\widehat{P}}^{\left(1\right)}\left[u^{\left(1\right)};a^{\left(1\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\\ \equiv {\left\{\mathsf{\delta }R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}.\end{array}$$

(54)

The expression obtained in Equation (54) no longer depends on the variations $\mathsf{\delta }\mathsf{\phi }$ and $\mathsf{\delta }\mathsf{\psi }$ but instead depends on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$. In particular, the expression in Equation (54) reveals that the sensitivities of the response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ to parameters that characterize the system’s boundary and/or internal interfaces can arise both from the direct effect and indirect effect terms. It also follows from Equation (54) that the quantity $\mathsf{\delta }R\left({\mathsf{\phi }}^0,{\mathsf{\psi }}^0;a^{\left(1\right)};{\mathsf{\alpha }}^0;\mathsf{\delta }\mathsf{\alpha }\right)$ has become a total differential in the parameter variation $\mathsf{\delta }\mathsf{\alpha }$ so it can be expressed in the following form:

$${\left\{\mathsf{\delta }R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}={\displaystyle \sum _{j_1=1}^{TP}{\left\{R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }{\mathsf{\alpha }}_{j_1}},$$

(55)

where the quantities $R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]\triangleq \partial R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]/\partial {\mathsf{\alpha }}_{j_1}$ represent the first-order sensitivities (i.e., partial functional derivatives) of the response with respect to the model parameter ${\mathsf{\alpha }}_{j_1}$, evaluated at the nominal values of the state functions and model parameters. Furthermore, each of the first-order sensitivities $R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]$ can be represented in integral form as follows:

$$R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]\triangleq {\displaystyle \prod _{k=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_k\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_k\left(\mathsf{\alpha }\right)}{\int }}d{x}_k{S}^{\left(1\right)} \left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]}}, j_1=1,\dots ,TP.$$

(56)

In particular, if the residual bilinear concomitant does not vanish, the functions $S^{\left(1\right)} \left[j_1;\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]$ would contain suitably defined Dirac delta functionals for expressing the respective non-zero boundary terms as volume integrals over the phase space of the independent variables. Dirac delta functionals would also be needed to represent, within $S^{\left(1\right)} \left[j_1;\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]$, the terms containing the derivatives of the boundary endpoints with respect to the model and/or response parameters.

Cacuci [36] has obtained the general expression of the arbitrarily high-order (n^th-order, where n is an arbitrary integer) sensitivities of the response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ to model parameters by treating each sensitivity as a response, applying the principles outlined in items (1)−(9) to successively higher-order sensitivities and, ultimately, using mathematical induction to demonstrate that general expression thus obtained for the n^th-order sensitivity is correct. Each of the (n − 1)^th-order sensitivities ${\partial }^{\left(n-1\right)}R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]/\partial {\mathsf{\alpha }}_{j_1}\dots \partial {\mathsf{\alpha }}_{j_{n-1}}\triangleq R^{\left(n-1\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;x\right);a^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1;x\right);\mathsf{\alpha }\right]$ of the response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ with respect to the parameters ${\mathsf{\alpha }}_{j_1}\dots {\mathsf{\alpha }}_{j_{n-1}}$, where $j_1,j_2,\dots ,j_{n-1}=1,\dots ,TP$, are used in the role of a “model response”, for which the first-order G-differential is obtained by applying the principles outlined in items (1)−(9), thereby ultimately obtaining the general expression of the n^th-order sensitivities of the response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$. The details of the n^th-CASAM-L methodology are provided by Cacuci [36]. The end results produced by the n^th-CASAM-L methodology enable the exact and efficient computation—overcoming the curse of dimensionality in sensitivity analysis—of the n^th-order partial sensitivities of the response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ with respect to the model parameters, using the expression provided below:

$$\begin{array}{l}{\left\{\mathsf{\delta }{R}^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }{u}^{\left(n\right)};\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}\triangleq {\left\{\frac{d}{d\mathsf{\epsilon }}{R}^{\left(n-1\right)}\left[u^{\left(n\right)}\left(x\right)+\mathsf{\epsilon }\mathsf{\delta }{u}^{\left(n\right)};{\mathsf{\alpha }}^0+\mathsf{\epsilon }\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{\mathsf{\epsilon }=0}\\ =-{\left\{{\widehat{P}}^{\left(n\right)}\left(a^{\left(n\right)};u^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}+{\left\{\frac{\partial }{\partial \mathsf{\alpha }}{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha }\right)}}\right\}}_{{\mathsf{\alpha }}^0}\mathsf{\delta }\mathsf{\alpha } \\ +{\left\{{\langle a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right),q_V^{\left(n\right)}\left[j_{n-2},\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right] \rangle }_n\right\}}_{{\mathsf{\alpha }}^0}\\ \equiv {\displaystyle \sum _{j_n=1}^{TP}{\left\{R^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n\right)};a^{\left(n\right)};\mathsf{\alpha }\right)\right\}}_{\left({\mathsf{\alpha }}^0\right)}}\mathsf{\delta }{\mathsf{\alpha }}_{j_n}={\displaystyle \sum _{j_n=1}^{TP}{\left\{\partial {}^{n}R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]/\partial {\mathsf{\alpha }}_{j_1}\dots \partial {\mathsf{\alpha }}_{j_n}\right\}}_{\left({\mathsf{\alpha }}^0\right)}}\mathsf{\delta }{\mathsf{\alpha }}_{j_n}.\end{array}$$

(57)

The various quantities which appear in Equation (57) are defined as follows:

(1)

The quantity $R^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n\right)};a^{\left(n\right)};\mathsf{\alpha }\right)\equiv \partial {}^{n}R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]/\partial {\mathsf{\alpha }}_{j_1}\dots \partial {\mathsf{\alpha }}_{j_n}$ denotes the n^th-order partial sensitivity of the response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ with respect to the model parameters, evaluated at the nominal values of the parameters and state functions. When the symmetries among the various partial derivatives/sensitivities are considered, the respective indices have ranges as follows: $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$; …; $j_n=1,\dots ,j_{n-1}$. In integral form, the n^th-order partial sensitivity of the response can be written as follows:

$$\begin{array}{l}{\partial }^{n}R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]/\partial {\mathsf{\alpha }}_{j_1}\dots \partial {\mathsf{\alpha }}_{j_n}\triangleq R^{\left(n\right)}\left[j_n,\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);\mathsf{\alpha }\right]\\ \triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\mathsf{\lambda }}_i\left(\mathsf{\alpha }\right)}{\overset{{\mathsf{\omega }}_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(n\right)}\left[j_n,\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);\mathsf{\alpha }\right]}},\end{array}$$

(58)

(2)

The n^th-level state $u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right)$, which appears in the arguments of $R^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n\right)};a^{\left(n\right)};\mathsf{\alpha }\right)$, is defined as follows:

$$\begin{array}{l}{u}^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right)\triangleq [\mathsf{\phi }\left(x\right);\mathsf{\psi }\left(x\right);a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right);\\ a_1^{\left(2\right)}\left(j_1;x\right),a_2^{\left(2\right)}\left(j_1;x\right),a_3^{\left(2\right)}\left(j_1;x\right),a_4^{\left(2\right)}\left(j_1;x\right),\dots ,\\ {a_1^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1\right),\dots ,a_k^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1\right),\dots ,a_{2^{n-1}}^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1\right)]}^{†}\end{array}$$

(59)

(3)

The inner product which appears in Equation (57) is defined in a Hilbert space denoted as ${\mathrm{H}}_n$, comprising $2^n$-dimensional block vectors ${\mathsf{\eta }}^{\left(n\right)}\left(x\right)\in {\mathrm{H}}_n$ having the following structure: ${\mathsf{\eta }}^{\left(n\right)}\left(x\right)\triangleq {\left[\dots ,{\mathsf{\eta }}_k^{\left(n\right)}\left(x\right),\dots \right]}^{†}$,${\mathsf{\eta }}_k^{\left(n\right)}\left(x\right)\triangleq {\left[{\mathsf{\eta }}_{k,1}^{\left(n\right)}\left(x\right),\dots ,{\mathsf{\eta }}_{k,TD}^{\left(n\right)}\left(x\right)\right]}^{†}$, $k=1,\dots ,2^n$. The inner product between two elements, ${\mathsf{\eta }}^{\left(n\right)}\left(x\right)\in {\mathrm{H}}_n$ and ${\mathsf{\xi }}^{\left(n\right)}\left(x\right)\in {\mathrm{H}}_n$, of the Hilbert space ${\mathrm{H}}_{n+1}$, is denoted as ${\langle {\mathsf{\eta }}^{\left(n\right)}\left(x\right),{\mathsf{\xi }}^{\left(n\right)}\left(x\right)\rangle }_n$ and is defined as follows:

$${\langle {\mathsf{\eta }}^{\left(n\right)}\left(x\right),{\mathsf{\xi }}^{\left(n\right)}\left(x\right)\rangle }_n\triangleq {\displaystyle \sum _{k=1}^{2^n}{\langle {\mathsf{\eta }}_k^{\left(n\right)}\left(x\right),{\mathsf{\xi }}_k^{\left(n\right)}\left(x\right)\rangle }_0}.$$

(60)

(4)

The adjoint operator ${\left[A^{\left(n\right)}\right]}_{2^n\times 2^n}$ which appears on the left side of Equation (62) is constructed by using the inner product defined in Equation (60), via the following relation:

$$\begin{array}{l}{\left\{{\langle a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right),\left[V^{\left(n\right)}\right]\mathsf{\delta }{u}^{\left(n\right)}\rangle }_n\right\}}_{{\mathsf{\alpha }}^0}\\ ={\left\{{\langle a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right),q_V^{\left(n\right)}\left[j_{n-2}\dots j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\rangle }_n\right\}}_{{\mathsf{\alpha }}^0}\\ ={\left\{{\langle \mathsf{\delta }{u}^{\left(n\right)}, A^{\left(n\right)}{a}^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right) \rangle }_n\right\}}_{{\mathsf{\alpha }}^0}+{\left\{P^{\left(n\right)}\left[\mathsf{\delta }{u}^{\left(n\right)};a^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0},\end{array}$$

(61)

(5)

The n^th-level adjoint sensitivity function $a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right)\triangleq {\left[\dots ,a_k^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right),\dots \right]}^{†}\in {\mathrm{H}}_n; k=1,\dots ,2^n$, which appears in the arguments of $R^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n\right)};a^{\left(n\right)};\mathsf{\alpha }\right)$, is the solution of the following n^th-level adjoint sensitivity system n^th-LASS, for $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$; …$j_{n-1}=1,\dots ,j_{n-2}$:

$${\left[A^{\left(n\right)}\right]}_{2^n\times 2^n}{a}^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right) =s_A^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right) ,$$

(62)

$${\left\{b_A^{\left(n\right)}\left[a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right),$$

(63)

where the vector $s_A^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right)\triangleq {\left[\dots ,s_k^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right),\dots \right]}^{†}$, $k=1,\dots ,2^n$ comprises $2^n$ components defined for each $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$; …; $j_{n-1}=1,\dots ,j_{n-2}$, as follows:

$$s_1^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right)\triangleq \frac{\partial S^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha }\right)}{\partial \mathsf{\phi }};$$

(64)

$$s_2^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right)\triangleq \frac{\partial S^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha }\right)}{\partial \mathsf{\psi }};$$

(65)

$$s_{2+k}^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right)\triangleq \frac{\partial S^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha }\right)}{\partial a_k^{\left(1\right)}}; k=1,2;$$

(66)

$$s_{2^2+k}^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right) \triangleq \frac{\partial S^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha }\right)}{\partial a_k^{\left(2\right)}\left(j_1\right)}; k=1,\dots ,2^2;$$

(67)

$$s_{2^3+k}^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right) \triangleq \frac{\partial S^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha }\right)}{\partial a_k^{\left(3\right)}\left(j_2;j_1\right)}; k=1,\dots ,2^3 ;$$

(68)

$$s_{2^{n-1}+k}^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathsf{\alpha }\right) \triangleq \frac{\partial S^{\left(n-1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha }\right)}{\partial a_k^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1\right)}; k=1,\dots ,2^{n-1}.$$

(69)

(6)

The quantity ${\left\{{\widehat{P}}^{\left(n\right)}\left(a^{\left(n\right)};u^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}$ denotes the residual bilinear concomitant defined on the phase-space boundary $x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right)$, evaluated at the nominal values of the model parameter and respective state functions.

$${\left[A^{\left(n\right)}\right]}_{2^n\times 2^n}\triangleq {\left[V^{\left(n\right)}\right]}_{2^n\times 2^n}^{*}={\left(\begin{array}{cc}{\left[V^{\left(n-1\right)}\right]}_{2^{n-1}\times 2^{n-1}}& {\left[0\right]}_{2^{n-1}\times 2^{n-1}}\\ {\left[V_{21}^{\left(n\right)}\right]}_{2^{n-1}\times 2^{n-1}}& {\left[V_{22}^{\left(n-1\right)}\right]}_{2^{n-1}\times 2^{n-1}}\end{array}\right)}^{*};$$

(70)

$$\begin{array}{l}{V}_{21}^{\left(n\right)}\triangleq \frac{\partial \left\{{\left[A^{\left(n-1\right)}\right]}_{2^{n-1}\times 2^{n-1}}{a}^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1;x\right)\right\}}{\partial u^{\left(n-1\right)}}\\ -\frac{\partial \left\{s_A^{\left(n-1\right)}\left[j_{n-2},\dots ,j_1;u^{\left(n-1\right)}\left(j_{n-3},\dots ,j_1;x\right);\mathsf{\alpha }\right]\right\}}{\partial u^{\left(n-1\right)}}; V_{22}^{\left(n-1\right)}\triangleq A^{\left(n-1\right)};\end{array}$$

(71)

$$\begin{array}{l}{q}_V^{\left(n\right)}\left[j_{n-2},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\triangleq \left(\begin{array}{c}{q}_V^{\left(n-2\right)}\left[j_{n-3},\dots ,j_1;\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\\ p^{\left(n-1\right)}\left[j_{n-2},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right]\end{array}\right)\\ \equiv {\left[\dots q_k^{\left(n\right)}\left(j_{n-2},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right)\dots \right]}^{†}, k=1,\dots ,2^n;\end{array}$$

(72)

$$\begin{array}{l}{p}^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1;u^{\left(n\right)};\mathsf{\alpha };\mathsf{\delta }\mathsf{\alpha }\right) \\ \triangleq \frac{\partial \left\{s_A^{\left(n-1\right)}\left[j_{n-2},\dots ,j_1;u^{\left(n-1\right)}\left(j_{n-3},\dots ,j_1;x\right);\mathsf{\alpha }\right]-A^{\left(n-1\right)}{a}^{\left(n-1\right)}\right\}}{\partial \mathsf{\alpha }}\mathsf{\delta }\mathsf{\alpha }.\end{array}$$

(73)

4. High-Order Sensitivity and Uncertainty Analysis of Nonlinear Models/Systems: Illustrative Example and General Theory

It is currently well known that a boiling water reactor (BWR) can undergo time-dependent power–amplitude oscillations, particularly when operated under high-power and low-flow conditions, due to the nonlinear coupling, through the void reactivity, between the neutronics and thermal-hydraulic processes. Such power oscillations can occur globally, when the reactor’s power oscillates in phase across the entire reactor core, or regionally, when part (e.g., half, for the so-called “first mode”) of the reactor core oscillates out of phase with respect to the other part (half), while the average power remains essentially constant across the reactor. In a pioneering sequence of works, March-Leuba, Cacuci, and Perez [45,46,47] proposed a phenomenological reduced-order model for analyzing qualitatively the dynamic behavior of BWRs, predicting global limit cycle oscillations in BWRs four years before they actually occurred [62] in an operating power reactor in the USA, on 9 March 1988, in the LaSalle County-2 BWR in Seneca, Illinois. The reduced-order model developed by March-Leuba, Cacuci, and Perez [45] comprises point neutron kinetics equations coupled nonlinearly to thermal-hydraulics equations that describe the time evolution of the fuel temperature and coolant density in the recirculation loop, which is treated as a single path of fluid of variable cross-sectional area but with constant mass flow, with the coolant entering the core at saturation temperature. This model showed that the major state variables, including power, reactivity, void fraction, and temperature, undergo limit cycle oscillations under high-power/low-flow operating conditions, and the amplitudes of these oscillations are very sensitive to the reactor’s operating conditions. This model also indicated that these limit cycle oscillations can become unstable—when increasing the heat transfer from the reactor to the coolant—and can undergo period-doubling bifurcations causing oscillatory (from periodic to aperiodic) time evolutions of the state variables (power, delayed neutron precursors, reactivity, coolant void fraction, and fuel temperature). The reduced-order model introduced by March-Leuba, Cacuci, and Perez [45] has generated great interest in the scientific community, inspiring investigations of the stability limits for in-phase and out-of-phase oscillation modes and the nature of the underlying bifurcations in the various state functions, as exemplified by [48,49,50,51,52,53,54,55].

Section 4.1 reviews the pioneering sensitivity and uncertainty analysis results presented in [43,44] for the reduced-order model of BWR dynamics developed by March-Leuba, Cacuci, and Perez [45]. This nonlinear model [45] illustrates generically the most extreme forms of nonlinear phenomena (from limit cycles to chaos) that can be displayed by nonlinear physical systems, so it is considered to be a representative paradigm for illustrating the application of the n^th-CASAM-N methodology [42] for computing efficiently and exactly the sensitivities of responses to parameters for nonlinear systems. For completeness, the general mathematical formalism underlying the n^th-CASAM-N [42] is reviewed in Section 4.2.

4.1. Illustrative Sensitivity and Uncertainty Analysis of a Reduced-Order Model of BWR-Dynamics

The reduced-order model developed by March-Leuba, Cacuci, and Perez [45] describes the fundamental processes underlying the dynamic behavior of a BWR when the reactor is perturbed from its originally critical steady-state condition, by insertion of reactivity. This reduced-order (lumped-parameter) model comprises differential equations that describe the following phenomena:

(a)

A point reactor representation of neutron kinetics, including the neutron precursors:

$$\frac{dn(t)}{dt}=n(t)\frac{\rho (t)-\beta }{\mathsf{\Lambda }}+\lambda c(t)+\frac{\rho (t)}{\mathsf{\Lambda }}$$

(74)

$$\frac{dc(t)}{dt}=n(t)\frac{\beta }{\mathsf{\Lambda }}-\lambda c(t)$$

(75)

(b)

A “one-node” lumped-parameter representation of the heat transfer process in the fuel:

$$\frac{dT(t)}{dt}=Q\left[n(t)+H(t)\Delta \right]-a_{3}T(t)$$

(76)

(c)

A “two-nodes” lumped-parameter representation of the channel thermal hydraulics, accounting for the reactivity feedback:

$$\frac{d^2\gamma (t)}{d{t}^2}+a_2\frac{d\gamma (t)}{dt}+a_1\gamma (t)=m{k}_{0}T(t)$$

(77)

$$\mathsf{\rho }(t)={\mathsf{\Gamma }}_1\mathsf{\gamma }(t)+{\mathsf{\Gamma }}_{2}T(t)$$

(78)

The initial conditions at $t=0$ for Equations (74)−(77) are as follows:

$$n(0)=n_0,$$

(79)

$$c(0)=c_0,$$

(80)

$$T(0)=T_0,$$

(81)

$$\gamma (0)={\gamma }_0.$$

(82)

The state variables in Equations (74)−(78) are defined as follows:

(i)

$n(t)\triangleq \left[N\left(t\right)-N_0\right]/N_0$ denotes the normalized excess neutron population, where $N\left(t\right)$ denotes the actual time-dependent neutron population and $N_0$ denotes the neutron population of the initially critical reactor.

(ii)

$c(t)\triangleq \left[C\left(t\right)-C_0\right]/C_0$ denotes the normalized excess population of delayed neutron precursors, where $C\left(t\right)$ denotes the actual time-dependent population of delayed neutron precursors and $C_0$ denotes the population of delayed neutron precursors of the initially critical reactor.

(iii)

$T(t)$ denotes the excess fuel temperature, defined as the departure from the temperature of the initially critical reactor.

(iv)

$H(t)$ denotes the Heaviside functional: $H(t)=0, for t<0$ and $H(t)=1, for t>0$.

(v)

$\gamma (t)$ denotes the relative excess coolant density, defined as the departure from the coolant density of the initially critical reactor.

(vi)

$\rho (t)$ denotes the excess reactivity, defined as the departure from the reactivity of the initially critical reactor.

The nominal values of the model parameters which appear in Equations (74)−(78) are reported in

Table 6. Nominal values of imprecisely known BWR model parameters.

Parameter Nr.	Parameter	Nominal Value	Unit of Measure
1	${\mathsf{\Gamma }}_1$	${\mathsf{\Gamma }}_1^0=0.15$	$-$
2	${\mathsf{\Gamma }}_2$	${\mathsf{\Gamma }}_2^0=-2.61\times {10}^{-5}$	$K^{-1}$
3	$\beta$	${\beta }^0=0.0056$	$-$
4	$\mathsf{\Lambda }$	${\mathsf{\Lambda }}^0=4.0\times {10}^{-5}$	$s$
5	$\lambda$	${\lambda }^0=0.08$	$s^{-1}$
6	$Q$	$Q^0=25.044$	$K/s$
7	$\Delta$	${\Delta }^0=-0.1$	$-$
8	$a_1$	$a_1^0=6.8166$	$s^{-2}$
9	$a_2$	$a_2^0=2.2494$	$s^{-1}$
10	$a_3$	$a_3^0=0.2325$	$s^{-1}$
11	$k_0$	$k_0^0=-0.01318$	$K^{-1}{s}^{-2}$
12	$n_0$	$n_0^0=0.0$	$-$
13	$c_0$	$c_0^0=0.0$	$-$
14	$T_0$	$T_0^0=0.0$	$-$
15	${\gamma }_0$	${\gamma }_0^0=0.0$	$K^{-1}$
16	${\gamma }_{10}$	${\gamma }_{10}^0=0.0$	$K^{-1}{s}^{-1}$

. Since the qualifier “excess” signifies that the above quantities are defined as departures from the critical reactor configuration, the nominal values of the initial conditions shown in Equations (79)−(82) are all zero at $t=0$.

The reduced-order model defined by Equations (74)−(82) has two equilibrium points in the phase space, one stable (denoted as “point A”) and the other one unstable (denoted as “point B”), at the following phase space locations:

• Stable equilibrium (A), which corresponds to the critical reactor configuration, which is defined as follows:

$$n_A=-\Delta ; c_A=-\frac{\mathsf{\beta }\Delta }{\mathsf{\lambda }\mathsf{\Lambda }}; T_A={\mathsf{\gamma }}_A={\mathsf{\rho }}_A=0.$$

(83)

b.

Unstable equilibrium (B):

$$\begin{array}{l}{n}_B=-1; c_B=-\frac{\mathsf{\beta }}{\mathsf{\lambda }\mathsf{\Lambda }}; T_B=\frac{Q\left(\Delta -1\right)}{a_3}; \\ {\mathsf{\gamma }}_B=kQ\left(\Delta -1\right); {\mathsf{\rho }}_B=\left(\frac{{\mathsf{\Gamma }}_{1}k}{a_3}+{\mathsf{\Gamma }}_2\right)\frac{Q\left(\Delta -1\right)}{a_3}.\end{array}$$

(84)

As discussed by March-Leuba, Cacuci, and Perez [45], when the feedback gain $m$ in Equation (77) is increased, thereby increasing the heat transfer from the reactor to the coolant, the equilibrium point A becomes unstable after the stability threshold is crossed. The phase-space trajectories of the lumped-parameter model oscillate between the two unstable equilibrium points until a stable limit cycle sets in. Increasing the value of the feedback gain $m$ beyond the stability of the first limit cycle will cause this cycle to bifurcate, through a period-doubling bifurcation, into a period-two cycle. Continuing to increase the feedback gain $m$ leads to a cascade of period-doubling bifurcations which lead to aperiodic behavior, as illustrated in Figure 3 by the bifurcation map for the excess temperature $T(t)$, which is obtained by plotting the extrema of $T(t)$ as a function of the feedback gain $m$. The values of the bifurcation variables are provided in

Table 7. Bifurcation points for the reduced-order BWR model by March-Leuba, Cacuci, and Perez (1984).

Bifurcation Order	Bifurcation Variable	Value	$\mathbf{Feedback} \mathbf{Gain} \left(\mathit{m}\right)$	${\mathit{\delta }}_{\mathit{j}}\triangleq \frac{{\mathit{k}}_{\mathit{j}+1}-{\mathit{k}}_{\mathit{j}}}{{\mathit{k}}_{\mathit{j}+2}-{\mathit{k}}_{\mathit{j}+1}}$
1	$k_{bif1}$	$1.70898·k_0$	$1.70898$	$-$
2	$k_{bif2}$	$2.66657·k_0$	$2.66657$	$-$
3	$k_{bif3}$	$2.86991·k_0$	$2.86991$	$4.7093$
4	$k_{bif4}$	$2.92027·k_0$	$2.92027$	$4.0375$
Transition to chaos (extrapolated)	$k_{chaos}$	$2.933995·k_0$	$2.933995$	$4.66921$

. The bifurcation map presented in Figure 3 for values of k ranging from $k=1.6·k_0$ to $k=3.6·k_0$ (i.e., $1.6\le m\le 3.6$) highlight the following Regions in the phase space:

• Region 1: Stable Region, before the first-order bifurcation, $m·k_0<k_{bif1}$, in which the state functions oscillate with decreasing amplitudes converging toward steady-state values.
• Region 2: Unstable Region between the first-order bifurcation and the second-order bifurcations, $k_{bif1}<m·k_0<k_{bif2}$, in which the state functions oscillate indefinitely between two unstable attractors.
• Region 3: Unstable Region between the second-order bifurcations and the third-order bifurcations, $k_{bif2}<m·k_0<k_{bif3}$, in which the state functions oscillate indefinitely between four unstable attractors.
• Region 4: Unstable Region between the third-order bifurcations and the fourth-order bifurcations, $k_{bif3}<m·k_0<k_{bif4}$, in which the state functions oscillate indefinitely between eight unstable attractors.
• Region 5: Chaotic Region stemming from the cascade of period-doubling pitchfork bifurcations produced when the feedback gain $m$ is increased past a critical value (called accumulation point), $m_c<m·k_0$, where $m_c=2.933995$, in which the state functions oscillate indefinitely between infinitely many unstable attractors.

The state functions are attracted in the stable Region 1 to the stable equilibrium attractor A, having the following coordinates in the phase space:

$$n_A=-0.1; c_A=175; T_A={\mathsf{\gamma }}_A={\mathsf{\rho }}_A=0.$$

(85)

The largest first-order sensitivity displayed by $T\left(t\right)$ is with respect to the initial condition ${\mathsf{\gamma }}_0$ for the excess coolant density $\mathsf{\gamma }(t)$, as depicted in Figure 4, which also presents, for comparison, the sensitivities of $T\left(t\right)$ to the other initial conditions. As depicted in Figure 4, the sensitivities oscillate with very large amplitudes immediately after the onset of the transient, which is initiated by perturbing the initially critical reactor in the steady state, and these amplitudes decay exponentially in time. The features depicted in Figure 4 are transmitted to the first-order total absolute standard deviation, denoted as $\mathsf{\sigma }$, induced in $T\left(t\right)$ by the sensitivities and parameter standard deviations, as shown in Figure 5.

The first-order total absolute standard deviation, denoted as $\mathsf{\sigma }$, for $T\left(t\right)$ is obtained by using the well-known formula $\mathsf{\sigma }=\sqrt{{\displaystyle \sum _{i=1}^{16}{\left(S_i{\mathsf{\sigma }}_i\right)}^2}}$, where $S_i\triangleq \partial n\left(t\right)/\partial {\mathsf{\alpha }}_i$ denotes the sensitivity of $T\left(t\right)$ to the parameter ${\mathsf{\alpha }}_i$, and where ${\mathsf{\sigma }}_i$ denotes the standard deviation for the respective parameter. The time evolution of the nominal value of the state function $T\left(t\right)$ is depicted in blue in Figure 5 along with the time evolution of the functions labeled “$T\left(t\right)\pm \mathsf{\sigma }$” and depicted in yellow, where $\mathsf{\sigma }$ denotes the total absolute standard deviation for $T\left(t\right)$. Immediately after the onset of the transient effected by perturbing the initially critical reactor in the steady state, the standard deviation for $T\left(t\right)$ is very large, reaching values that more than 10 times larger than the state function $T\left(t\right)$ itself. In time, the standard deviation decays exponentially, reaching values comparable to the state function $T\left(t\right)$ after about 140 s from the start of the transient. Very importantly, Figure 5 indicates that predictions of the excess temperature of $T\left(t\right)$ during the first 120 s after the BWR model is perturbed from its initial critical steady state would be unreliable. Furthermore, it is possible that that the model could be “pushed” into one of the oscillatory Regions by the large amplitudes reached by the standard deviation of $T\left(t\right)$ immediately after the onset of the perturbation.

The largest first-order sensitivity displayed by $T\left(t\right)$ in the oscillatory Region 2 is also with respect to the initial condition ${\gamma }_0$; the time evolution of this sensitivity is depicted in Figure 6, together with the sensitivities of $T\left(t\right)$ to the other initial conditions. As depicted in Figure 6, the amplitudes of these sensitivities are about two orders of magnitude larger than the corresponding first-order sensitivities. In contradistinction with the decaying amplitudes of the oscillations in Region 1, the oscillation amplitudes of the sensitivities in Region 2 stabilize to large values asymptotically in time. The features depicted in Figure 6 are transmitted to the first-order total absolute standard deviation induced in $T\left(t\right)$, as shown in Figure 7, which depicts (in blue) the time evolution of the nominal value of the state function $T\left(t\right)$ along with the time evolution (in yellow) of the functions $T\left(t\right)\pm \mathsf{\sigma }$. As indicated in Figure 7, the standard deviation increases rapidly after the onset of the transient (by perturbing the initially critical reactor in the steady state), oscillating with rapidly increasing amplitudes, reaching in 40 s after the start of the transient values that are 400K larger than the amplitude of the excess temperature, $T\left(t\right)$, which oscillates around the initial value $T_0=0K$. It is evident from Figure 7 that the standard deviation of $T\left(t\right)$ renders the prediction of the evolution of this state function highly unreliable after less than 10 s from the initiation of the transient.

The evolution of the standard deviation of $T\left(t\right)$ in the oscillatory Region 3 is depicted in Figure 8, which indicates that a standard deviation as small as 5% in the model’s parameters and initial conditions will be magnified in the oscillatory regions so much as to preclude any accurate prediction of the actual values of the excess temperature $T\left(t\right)$. This conclusion is also evidenced by the results presented in

Table 8. Maxima and minima of the uncertainty of $T(t)$ considering a 5% std. dev. for all parameters and initial conditions in Region 5 for various time windows.

Time Windows	Minimum Sensitivity Value	Maximum Sensitivity Value
0–25 s	−1.26 × 1008	1.26 × 1008
25–50 s	−3.46 × 1015	3.41 × 1015
50–75 s	−1.44 × 1022	1.44 × 1022
75–100 s	−2.53 × 1029	2.51 × 1029
100–125 s	−1.45 × 1035	1.47 × 1035
125–150 s	−4.59 × 1043	4.56 × 1043

, which presents the largest positive and negative values attained by the oscillation amplitude of $T\left(t\right)$ in the aperiodic Region 5, for 6 windows of 25 s each, covering the transient period from 0 to 150 s.

4.2. Overview of the n^th-Order Comprehensive Sensitivity Analysis Methodology for Nonlinear Systems (n^th-CASAM-N)

Using the same notation for the model’s independent variables and parameters as for the linear system considered in Section 3, the mathematical modeling of a nonlinear physical system can be represented in operator form as follows:

$$N\left[u\left(x\right),\mathsf{\alpha }\right]=Q\left(x,\mathsf{\alpha }\right) , x\in {\mathsf{\Omega }}_x\left(\mathsf{\alpha }\right),$$

(86)

The quantities which appear in Equation (86) are defined as follows:

• $u\left(x\right)\triangleq {\left[u_1\left(x\right),\dots ,u_{TD}\left(x\right)\right]}^{†}$ is a $TD$-dimensional column vector of dependent variables; the abbreviation “$TD$” denotes “total number of dependent variables.” The functions $u_i\left(x\right), i=1, \dots , TD$ denote the system’s “dependent variables” (also called “state functions”); $u\left(x\right)\in {\mathrm{E}}_u$, where ${\mathrm{E}}_u$ is a normed linear space over the scalar field $F$ of real numbers.
• $N\left[u\left(x\right);\mathsf{\alpha }\right]\triangleq {\left[N_1\left(u;\mathsf{\alpha }\right),\dots ,N_{TD}\left(u;\mathsf{\alpha }\right)\right]}^{†}$ denotes a $TD$-dimensional column vector. The components $N_i\left(u;\mathsf{\alpha }\right), i=1,\dots ,TD$ are operators (including differential, difference, integral, distributions, and/or finite or infinite matrices) acting (usually) nonlinearly on the dependent variables $u\left(x\right)$, the independent variables $x$, and the model parameters $\mathsf{\alpha }$. The mapping $N\left(u;\mathsf{\alpha }\right)$ is defined on the combined domains of the model’s parameters and state functions, i.e., $N: D\subset \mathrm{E}\to {\mathrm{E}}_Q$, where $D=D_u\oplus D_{\alpha }$, $D_u\subset {\mathrm{E}}_u$, $D_{\alpha }\subset {\mathrm{E}}_{\alpha }$, $\mathrm{E}={\mathrm{E}}_u\oplus {\mathrm{E}}_{\alpha }$.
• $Q\left(x,\mathsf{\alpha }\right) \triangleq {\left[q_1\left(x;\mathsf{\alpha }\right),\dots ,q_{TD}\left(x;\mathsf{\alpha }\right)\right]}^{†}$ is a $TD$-dimensional column vector which represents inhomogeneous source terms, which usually depend nonlinearly on the uncertain parameters $\mathsf{\alpha }$. The vector $Q\left(x,\mathsf{\alpha }\right)$ is defined on a normed linear space denoted as ${\mathrm{E}}_Q$, i.e., $Q\in {\mathrm{E}}_Q$.

Boundary and/or initial conditions must also be provided if differential operators appear in Equation (86). In operator form, these boundaries and/or initial conditions are represented as follows:

$$B\left[u\left(x\right);\mathsf{\alpha }; x\right]-C\left(x,\mathsf{\alpha }\right)=0, x\in \partial {\mathsf{\Omega }}_x\left(\mathsf{\alpha }\right),$$

(87)

Solving Equations (86) and (87) at the nominal parameter values ${\mathsf{\alpha }}^0\triangleq {\left[{\alpha }_1^0,\dots ,{\alpha }_i^0,\dots ,{\alpha }_{TP}^0\right]}^{†}$ provides the “nominal solution” $u^0\left(x\right)$, i.e., the vectors $u^0\left(x\right)$ and ${\mathsf{\alpha }}^0$, satisfies the following equations:

$$N\left[u^0\left(x\right);{\mathsf{\alpha }}^0\right]=Q\left(x,{\mathsf{\alpha }}^0\right) , x\in {\mathsf{\Omega }}_x,$$

(88)

$$B\left[u^0\left(x\right);{\mathsf{\alpha }}^0; x\right]-C\left(x,{\mathsf{\alpha }}^0\right)=0, x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right).$$

(89)

The model response is a functional considered to be representable in the following generic integral form:

$$R\left[u\left(x\right);\mathsf{\alpha }\right]\triangleq {\displaystyle \underset{{\lambda }_1\left(\mathsf{\alpha }\right)}{\overset{{\omega }_1\left(\mathsf{\alpha }\right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\mathsf{\alpha }\right)}{\overset{{\omega }_{TI}\left(\mathsf{\alpha }\right)}{\int }}S\left[u\left(x\right);\mathsf{\alpha };x\right]d{x}_1\dots d{x}_{TI}}},$$

(90)

where $S\left[u\left(x\right);\mathsf{\alpha }\right]$ is a suitably differentiable nonlinear function of $u\left(x\right)$ and of $\mathsf{\alpha }$. It is important to note that the components of $\mathsf{\alpha }$ include not only parameters that appear in the equations defining the computational model per se, i.e., in Equations (86) and (87), but also include parameters that specifically occur only in the definition of the response under consideration. It is also important to note that the system’s definition domain, $\mathsf{\Omega }\left(\mathsf{\alpha }\right)$, in phase space is considered to be imprecisely known, subject to uncertainties in the components of the vector of model parameters $\mathsf{\alpha }$. Therefore, the system domain’s boundary, $\partial \mathsf{\Omega }\left(\mathsf{\alpha }\right)$, as well as the model response $R\left[u\left(x\right);\mathsf{\alpha }\right]$, will be affected by the boundary uncertainties that affect the endpoints ${\lambda }_i\left(\mathsf{\alpha }\right), {\omega }_i\left(\mathsf{\alpha }\right), i=1,\dots ,TI$. Such boundary uncertainties stem most often from manufacturing uncertainties.

Variation $\delta \mathsf{\alpha }$ in the model and boundary parameters will cause corresponding variations $v^{\left(1\right)}\left(x\right)\triangleq {\left[\delta u_1\left(x\right),\dots ,\delta u_{TD}\left(x\right)\right]}^{†}$ around the nominal solution $u^0\left(x\right)$ in the forward state functions. The variations $\delta \mathsf{\alpha }$ and $v^{\left(1\right)}\left(x\right)$ will, in turn, induce variations in the system’s response. The first-order G-differential $\delta R\left(e^0;h\right)\equiv {\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };v^{\left(1\right)}\left(x\right);\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ is obtained, by definition, as follows:

$$\begin{array}{l}{\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };v^{\left(1\right)}\left(x\right);\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\triangleq \left\{\frac{d}{d\epsilon }{\displaystyle \underset{{\lambda }_1\left({\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right)}{\overset{{\omega }_1\left({\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left({\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right)}{\overset{{\omega }_{TI}\left({\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right)}{\int }}S\left(u^0+\epsilon v^{\left(1\right)},{\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha };x\right)d{x}_1\dots d{x}_{TI}}}\right\}\\ \triangleq {\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{dir}+{\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };v^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}.\end{array}$$

(91)

In Equation (91), the direct effect term ${\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{dir}$ comprises only dependencies on $\delta \mathsf{\alpha }$ and is defined as follows:

$${\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{dir}\triangleq {\left\{\frac{\partial R\left(u;\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}\right\}}_{{\mathsf{\alpha }}^0}\delta \mathsf{\alpha }\triangleq {\displaystyle \sum _{j_1=1}^{TP}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);\mathsf{\alpha }\right]\right\}}_{dir}\delta {\alpha }_{j_1}},$$

(92)

where

$$\begin{array}{l}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);\mathsf{\alpha }\right]\right\}}_{dir}\triangleq {\left\{{\displaystyle \underset{{\lambda }_1\left(\mathsf{\alpha }\right)}{\overset{{\omega }_1\left(\mathsf{\alpha }\right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\mathsf{\alpha }\right)}{\overset{{\omega }_{TI}\left(\mathsf{\alpha }\right)}{\int }}\frac{\partial S\left(u;\mathsf{\alpha };\mathsf{\alpha }\right)}{\partial {\alpha }_{j_1}}d{x}_1\dots d{x}_{TI}}}\right\}}_{{\mathsf{\alpha }}^0}\\ +{\displaystyle \sum _{j=1}^{TI}{\left\{\frac{\partial {\omega }_j\left(\mathsf{\alpha }\right)}{\partial {\alpha }_{j_1}}{\displaystyle \underset{{\lambda }_1}{\overset{{\omega }_1}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{j-1}}{\overset{{\omega }_{j-1}}{\int }}{\displaystyle \underset{{\lambda }_{j+1}}{\overset{{\omega }_{j+1}}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}}{\overset{{\omega }_{TI}}{\int }}d{x}_{TI}S\left[u\left(x_1 ,.,{\omega }_j\left(\mathsf{\alpha }\right),.,x_{N_x}\right);\mathsf{\alpha }\right]}}}}\right\}}_{{\mathsf{\alpha }}^0}} \\ -{\displaystyle \sum _{j=1}^{TI}{\left\{\frac{\partial {\lambda }_j\left(\mathsf{\alpha }\right)}{\partial {\alpha }_{j_1}}{\displaystyle \underset{{\lambda }_1}{\overset{{\omega }_1}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{j-1}}{\overset{{\omega }_{j-1}}{\int }}{\displaystyle \underset{{\lambda }_{j+1}}{\overset{{\omega }_{j+1}}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}}{\overset{{\omega }_{TI}}{\int }}d{x}_{TI}S\left[u\left(x_1 ,.,{\lambda }_j\left(\mathsf{\alpha }\right),.,x_{N_x}\right);\mathsf{\alpha }\right]}}}}\right\}}_{{\mathsf{\alpha }}^0}}.\end{array}$$

(93)

On the other hand, the quantity ${\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };v^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}$ in Equation (91) comprises only variations in the state functions and is therefore called the indirect effect term, having the following expression:

$$\begin{array}{l}{\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };v^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}\triangleq {\displaystyle \underset{{\lambda }_1\left({\mathsf{\alpha }}^0\right)}{\overset{{\omega }_1\left({\mathsf{\alpha }}^0\right)}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{TI}\left({\mathsf{\alpha }}^0\right)}{\overset{{\omega }_{TI}\left({\mathsf{\alpha }}^0\right)}{\int }}d{x}_{TI}{\left\{ \frac{\partial S\left(u;\mathsf{\alpha };x\right)}{\partial u}{v}^{\left(1\right)}\left(x\right)\right\}}_{{\mathsf{\alpha }}^0}}};\\ \frac{\partial [ ]}{\partial u}{v}^{\left(1\right)}\left(x\right)\triangleq {\displaystyle \sum _{i=1}^{TD}\frac{\partial [ ]}{\partial u_i\left(x\right)}\delta u_i\left(x\right)}.\end{array}$$

(94)

The indirect effect term induces variations in the response through the variations in the state functions, which are, in turn, caused by the parameter variations through the equations underlying the model. Evidently, the indirect effect term can be quantified only after having determined the variation $v^{\left(1\right)}\left(x\right)$ in terms of the variation $\delta \mathsf{\alpha }$. The indirect effect term induces variations in the response through the variations in the state functions, which are, in turn, caused by the parameter variations through the equations underlying the model. Evidently, the indirect effect term can be quantified only after having determined the variation $v^{\left(1\right)}\left(x\right)$ in terms of the variation $\delta \mathsf{\alpha }$.

The first-order relationship between the vectors $v^{\left(1\right)}\left(x\right)$ and $\delta \mathsf{\alpha }$ is determined by solving the equations obtained by applying the definition of the G-differential to Equations (86) and (87), which yields the following equations:

$${\left\{\frac{d}{d\epsilon }N\left[u^0+\epsilon v^{\left(1\right)}\left(x\right);{\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right]\right\}}_{\mathsf{\epsilon }=0}={\left\{\frac{d}{d\epsilon }Q\left(x,{\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right)\right\}}_{\epsilon =0},$$

(95)

$${\left\{\frac{d}{d\epsilon }B\left[u^0+\epsilon v^{\left(1\right)}\left(x\right);{\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right]\right\}}_{\epsilon =0}-{\left\{\frac{d}{d\epsilon }C\left(x,{\mathsf{\alpha }}^0+\epsilon \delta \mathsf{\alpha }\right)\right\}}_{\epsilon =0}=0.$$

(96)

Carrying out the differentiations with respect to $\epsilon$ in Equations (95) and (96) and setting $\epsilon =0$ in the resulting expressions yields the following equations:

$${\left\{N^{\left(1\right)}\left(u;\mathsf{\alpha }\right)v^{\left(1\right)}\left(x\right)\right\}}_{{\mathsf{\alpha }}^0}={\left\{q_V^{\left(1\right)}\left(u;\mathsf{\alpha };\delta \mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}, x\in {\mathsf{\Omega }}_x,$$

(97)

$${\left\{b_V^{\left(1\right)}\left(u;\mathsf{\alpha };v^{\left(1\right)} ;\delta \mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right).$$

(98)

In Equations (97) and (98), the superscript “(1)” indicates “1^st-Level” and the various quantities which appear in these equations are defined as follows:

$$N^{\left(1\right)}\left(u;\mathsf{\alpha }\right)\triangleq \left\{\frac{\partial N\left( u;\mathsf{\alpha }\right)}{\partial u}\right\}\triangleq \left(\begin{array}{ccc}\frac{\partial N_1}{\partial u_1}& \dots & \frac{\partial N_1}{\partial u_{TD}}\\ ⋮& \ddots & ⋮\\ \frac{\partial N_{TD}}{\partial u_1}& \cdots & \frac{\partial N_{TD}}{\partial u_{TD}}\end{array}\right) ;$$

(99)

$$q_V^{\left(1\right)}\left(u;\mathsf{\alpha };\delta \mathsf{\alpha }\right)\triangleq \frac{\partial \left[Q\left(\mathsf{\alpha }\right)-N\left( u;\mathsf{\alpha }\right)\right]}{\partial \mathsf{\alpha }} \delta \mathsf{\alpha }\triangleq {\displaystyle \sum _{j_1=1}^{TP}{s}_V^{\left(1\right)}\left(j_1;u;\mathsf{\alpha }\right)\delta {\alpha }_{j_1}};$$

(100)

$$s_V^{\left(1\right)}\left(j_1;u;\mathsf{\alpha }\right)\triangleq \frac{\partial \left[Q\left(\mathsf{\alpha }\right)-N\left( u;\mathsf{\alpha }\right)\right]}{\partial {\alpha }_{j_1}};$$

(101)

$${\left\{b_V^{\left(1\right)}\left(u;\mathsf{\alpha };v^{\left(1\right)};\delta \mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}\triangleq {\left\{\frac{\partial B\left(u;\mathsf{\alpha }\right)}{\partial u}\right\}}_{{\mathsf{\alpha }}^0}{v}^{\left(1\right)}+{\left\{\frac{\partial \left[B\left(u;\mathsf{\alpha }\right)-C\left(\mathsf{\alpha }\right)\right]}{\partial \mathsf{\alpha }}\right\}}_{{\mathsf{\alpha }}^0} \delta \mathsf{\alpha }.$$

(102)

The system comprising Equations (97) and (98) is called the “1^st-Level Variational Sensitivity System” (1^st-LVSS). In order to determine the solutions of the 1^st-LVSS that would correspond to every parameter variation $\delta {\alpha }_{j_1}$, $j_1=1, \dots , TP$, the 1^st-LVSS would need to be solved $TP$ times, with distinct right sides for each $\delta {\alpha }_{j_1}$, thus requiring $TP$ large-scale computations.

The need for computing the vector $v^{\left(1\right)}\left(x\right)$ is eliminated by applying the 1^st-CASAM-N to express the indirect effect term defined in Equation (94) in terms of the solutions of the “1^st-Level Adjoint Sensitivity System” (1^st-LASS), the construction of which requires the introduction of adjoint operators. The 1^st-CASAM-N represents a generalization of the pioneering works by Cacuci (1981a, b) which had conceived the “Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” but in which the domain boundary was considered to have been perfectly well-known, free of uncertainties. The qualifier “comprehensive” is meant to indicate that that the 1^st-CASAM-N considers that the internal and external boundaries $\partial {\mathsf{\Omega }}_x\left(\mathsf{\alpha }\right)$ of the phase-space domain depend on the uncertain model parameters $\mathsf{\alpha }$ and are thereby imprecisely known, subject to uncertainties. The construction of the “1^st-Level Adjoint Sensitivity System” (1^st-LASS) is accomplished by introducing a real Hilbert space denoted as ${\mathrm{H}}_1\left({\mathsf{\Omega }}_x\right)$, endowed with an inner product of two vectors $u^{(a)}\left(x\right)\in {\mathrm{H}}_1$ and $u^{(b)}\left(x\right)\in {\mathrm{H}}_1$ denoted as ${\langle u^{(a)},u^{(b)}\rangle }_1$ and defined as follows:

$${\langle u^{(a)},u^{(b)}\rangle }_1\triangleq {\left\{{\displaystyle \underset{{\lambda }_1\left(\mathsf{\alpha }\right)}{\overset{{\omega }_1\left(\mathsf{\alpha }\right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\mathsf{\alpha }\right)}{\overset{{\omega }_{TI}\left(\mathsf{\alpha }\right)}{\int }}\left[u^{(a)}\left(x\right)u^{(b)}\left(x\right)\right]d{x}_1\dots d{x}_{TI}}}\right\}}_{{\mathsf{\alpha }}^0},$$

(103)

where $u^{(a)}\left(x\right)u^{(b)}\left(x\right) \triangleq {\displaystyle \sum _{i=1}^{TD}{u}_i^{(a)}\left(x\right)u_i^{(b)}}\left(x\right)$, and where the “dagger” ($†$), which indicates “transposition”, has been omitted (to simplify the notation) in the representation of this scalar product. Using the definition of the adjoint operator in ${\mathrm{H}}_1\left({\mathsf{\Omega }}_x\right)$ and applying the 1^st-CASAM-N yields the following expression for the first G-differential of the response $R\left[u\left(x\right);\mathsf{\alpha }\right]$:

$${\left\{\delta R\left[u\left(x\right);\mathsf{\alpha };v^{\left(1\right)}\left(x\right);\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\triangleq {\displaystyle \sum _{j_1=1}^{TP}{\left\{R^{\left(1\right)}\left[j_1;U^{\left(1\right)}\left(2^0;x\right);A^{(1)}\left(2^0;x\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\delta {\alpha }_{j_1}},$$

(104)

where for each $j_1=1, \dots , TP$, the quantity $R^{\left(1\right)}\left[j_1;U^{\left(1\right)}\left(2^0;x\right);A^{(1)}\left(2^0;x\right);\mathsf{\alpha }\right]$ denotes the first-order sensitivities of the response $R\left[u\left(x\right);\mathsf{\alpha }\right]$ with respect to the model parameters ${\alpha }_{j_1}$ and has the following expression:

$$\begin{array}{l}{R}^{\left(1\right)}\left[j_1;U^{\left(1\right)}\left(2^0;x\right);A^{(1)}\left(2^0;x\right);\mathsf{\alpha }\right]=\frac{\partial R\left[u\left(x\right);\mathsf{\alpha }\right]}{\partial {\alpha }_{j_1}}\\ =-\frac{{\left[\partial {\widehat{P}}^{\left(1\right)}\left(U^{\left(1\right)};A^{(1)};\delta \mathsf{\alpha }\right)\right]}_{\partial {\mathsf{\Omega }}_x}}{\partial {\alpha }_{j_1}}+{\langle A^{\left(1\right)}\left(2^0;x\right),s_V^{\left(1\right)}\left(j_1;u;\mathsf{\alpha }\right)\rangle }_1 .\end{array}$$

(105)

The quantity $s_V^{\left(1\right)}\left(j_1;u;\mathsf{\alpha }\right)$, which appears in the inner product on the right side of Equation (105), is defined as follows:

$$s_V^{\left(1\right)}\left(j_1;u;\mathsf{\alpha }\right)\triangleq \partial \left[Q\left(\mathsf{\alpha }\right)-N\left(u;\mathsf{\alpha }\right)\right]/\partial {\alpha }_{j_1}.$$

(106)

The first-level adjoint sensitivity function $A^{\left(1\right)}\left(2^0;x\right)$, which appears in the inner product on the right side of Equation (105), is the solution of the following 1^st-LASS:

$$\begin{array}{l}{\left\{A{M}^{\left(1\right)}\left[2^0\times 2^0;U^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha }\right]A^{\left(1\right)}\left(2^0;x\right)\right\}}_{{\mathsf{\alpha }}^0}\\ ={\left\{Q_A^{\left(1\right)}\left[2^0;U^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}, x\in {\mathsf{\Omega }}_x,\end{array}$$

(107)

$${\left\{B_A^{\left(1\right)}\left[2^0;U^{\left(1\right)}\left(2^0;x\right);V^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right).$$

(108)

In Equations (107) and (108), the superscript “(1)” indicates “1^st-Level”, $2^0\equiv 1$ (evidently), and the various quantities which appear in these equations are defined as follows:

$$A{M}^{\left(1\right)}\left[2^0\times 2^0;U^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha }\right]\equiv {\left[N^{\left(1\right)}\left(u;\mathsf{\alpha }\right)\right]}^{*}\equiv {\left[\partial N\left(u;\mathsf{\alpha }\right)/\partial u\right]}^{*};U^{\left(1\right)}\left(2^0;x\right)\equiv u\left(x\right);$$

(109)

$$s_V^{\left(1\right)}\left(j_1;u;\mathsf{\alpha }\right)\triangleq \partial \left[Q\left(\mathsf{\alpha }\right)-N\left( u;\mathsf{\alpha }\right)\right]/\partial {\alpha }_{j_1};$$

(110)

$$Q_A^{\left(1\right)}\left[2^0;U^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\equiv q_A^{\left(1\right)}\left[u\left(x\right);\mathsf{\alpha }\right]\triangleq {\left\{\partial S\left(u;\mathsf{\alpha } \right)/\partial u\right\}}_{{\mathsf{\alpha }}^0}.$$

(111)

The domain of the first-level adjoint operator $A{M}^{\left(1\right)}\left[2^0\times 2^0;U^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha }\right]$ is determined by selecting adjoint boundary and/or initial conditions, represented in operator form in Equation (108), such as to eliminate boundary terms containing unknown values of the function $v^{\left(1\right)}\left(x\right)$ which arise in the bilinear concomitant ${\left\{{\left[P^{\left(1\right)}\left(u;\mathsf{\alpha }; A^{(1)};v^{\left(1\right)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}$ in the course of constructing $A{M}^{\left(1\right)}\left[2^0\times 2^0;U^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha }\right]$ through the following relation:

$$\begin{array}{l}{\left\{{\langle A^{\left(1\right)}\left(2^0;x\right), N^{\left(1\right)}\left(u;\mathsf{\alpha }\right)v^{\left(1\right)}\rangle }_1\right\}}_{{\mathsf{\alpha }}^0}={\left\{{\langle A^{\left(1\right)}\left(2^0;x\right), q_V^{\left(1\right)}\left(u;\mathsf{\alpha };\delta \mathsf{\alpha }\right)\rangle }_1\right\}}_{{\mathsf{\alpha }}^0}\\ ={\left\{{\langle A{M}^{\left(1\right)}\left[2^0\times 2^0;U^{\left(1\right)}\left(2^0;x\right);\mathsf{\alpha }\right]A^{\left(1\right)}\left(2^0;x\right), v^{\left(1\right)}\rangle }_1\right\}}_{{\mathsf{\alpha }}^0}+{\left\{{\left[P^{\left(1\right)}\left(u;\mathsf{\alpha };A^{(1)} ;v^{\left(1\right)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0},\end{array}$$

(112)

where ${\left\{{\left[P^{\left(1\right)}\left(u;\mathsf{\alpha }; A^{(1)};v^{\left(1\right)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}$ denotes the associated bilinear concomitant evaluated on the space/time domain’s boundary $\partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right)$. The selection of the adjoint boundary conditions represented by Equation (108) reduces the bilinear concomitant ${\left\{{\left[P^{\left(1\right)}\left(u;\mathsf{\alpha }; A^{(1)};v^{\left(1\right)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}$ to a quantity denoted as ${\left\{{\left[{\widehat{P}}^{\left(1\right)}\left(u;\mathsf{\alpha }; A^{(1)};\mathsf{\delta }\mathsf{\alpha }\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}$, which will contain boundary terms involving only known values of $\mathsf{\delta }\mathsf{\alpha }$, ${\mathsf{\alpha }}^0$, $u^0$, and $A^{(1)}$.

As indicated by Equation (105), each of the first-order sensitivities $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);\mathsf{\alpha }\right]$ of the response $R\left[u\left(x\right);\mathsf{\alpha }\right]$ with respect to the model parameters ${\mathsf{\alpha }}_{j_1}$ (including boundary and initial conditions) can be computed inexpensively after having obtained the function $A^{(1)}\left(x\right)\in H_1$, using just quadrature formulas to evaluate the various inner products involving $A^{(1)}\left(x\right)$. The function $A^{(1)}\left(x\right)$ is obtained by solving numerically the 1^st-LASS, cf. Equations (107) and (108), which is the only large-scale computation needed for obtaining all of the first-order sensitivities, since the 1^st-LASS is independent of parameter variation $\delta {\alpha }_{j_1}$, $j_1=1, \dots , TP$, and therefore needs to be solved only once, regardless of the number of model parameters under consideration. Furthermore, since the 1^st-LASS is linear in $A^{(1)}\left(x\right)$, solving it requires less computational effort than solving the original model, which is nonlinear in $u\left(x\right)$. The first-order sensitivity $R^{\left(1\right)}\left[j_1;U^{\left(1\right)}\left(2^0;x\right);A^{(1)}\left(2^0;x\right);\mathsf{\alpha }\right]$ is used in the role of a “model response” for determining the second- and higher-order sensitivities of the response with respect to the model’s parameters.

The generic expression of the n^th-order sensitivity $R^{\left(n\right)}\left[j_n;\dots ;j_1;U^{\left(n\right)};A^{(n)};\mathsf{\alpha }\right]\equiv {\partial }^{n}R\left[u\left(x\right);\mathsf{\alpha }\right]/\partial {\alpha }_{j_n}\dots \partial {\alpha }_{j_1}$ of the response $R\left[u\left(x\right);\mathsf{\alpha }\right]$ with respect to the parameters ${\alpha }_{j_1},\dots ,{\alpha }_{j_n}$ is obtained, for each $j_1,\dots ,j_n=1,\dots ,TP$, by applying the n^th-Order Comprehensive Sensitivity Analysis Methodology for Nonlinear Systems (n^th-CASAM-N) conceived by Cacuci [40]. Using mathematical induction, Cacuci [40] obtained the n^th-order sensitivity $R^{\left(n\right)}\left[j_n;\dots ;j_1;U^{\left(n\right)};A^{(n)};\mathsf{\alpha }\right]\equiv {\partial }^{n}R\left[u\left(x\right);\mathsf{\alpha }\right]/\partial {\alpha }_{j_n}\dots \partial {\alpha }_{j_1}$ from the expression of the total first-order G-differential of each of the (n − 1)^th-order sensitivities, each of which was shown by Cacuci [40] to have the following expression for $j_1,\dots ,j_{n-1}=1,\dots ,TP$:

$$\begin{array}{l}{\left\{\delta R^{\left(n-1\right)}\left[j_{n-1};\dots ;j_1;U^{\left(n-1\right)};A^{(n-1)};\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\\ ={\left\{{\langle A^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right),Q_V^{\left(n\right)}\left(2^{n-1};U^{\left(n\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right)\rangle }_n\right\}}_{{\mathsf{\alpha }}^0}\\ -{\left\{{\left[{\widehat{P}}^{\left(n\right)}\left(U^{\left(n\right)};A^{(n)};\delta \mathsf{\alpha }\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}+{\displaystyle \sum _{j_n=1}^{TP}{\left\{\frac{\partial R^{\left(n-1\right)}\left[j_{n-1};\dots ;j_1;U^{\left(n-1\right)};A^{(n-1)};\mathsf{\alpha }\right]}{\partial {\alpha }_{j_n}}\right\}}_{{\mathsf{\alpha }}^0}}\delta {\alpha }_{j_n}\\ \triangleq {\displaystyle \sum _{j_n=1}^{TP}{\left\{{\partial }^{n}R\left[u\left(x\right);\mathsf{\alpha }\right]/\partial {\alpha }_{j_n}\dots \partial {\alpha }_{j_1}\right\}}_{{\mathsf{\alpha }}^0}}\delta {\alpha }_{j_n},\end{array}$$

(113)

where

$$\begin{array}{l}{R}^{\left(n\right)}\left[j_n;\dots ;j_1;U^{\left(n\right)};A^{(n)};\mathsf{\alpha }\right]\equiv {\partial }^{n}R\left[u\left(x\right);\mathsf{\alpha }\right]/\partial {\alpha }_{j_n}\dots \partial {\alpha }_{j_1}=\\ \frac{\partial R^{\left(n-1\right)}\left[j_{n-1};\dots ;j_1;U^{\left(n-1\right)};A^{(n-1)};\mathsf{\alpha }\right]}{\partial {\alpha }_{j_n}}-\frac{{\left[\partial {\widehat{P}}^{\left(n\right)}\left(U^{\left(n\right)};A^{(n)};\delta \mathsf{\alpha }\right)\right]}_{\partial {\mathsf{\Omega }}_x}}{\partial {\alpha }_{j_n}}\\ + {\displaystyle \sum _{i=1}^{2^{n-1}}{\langle a^{(n)}\left(i;j_{n-1};\dots ;j_1;x\right),s_V^{\left(n\right)}\left[i;j_n;\dots ;j_1;U^{\left(n\right)}\left(x\right);\mathsf{\alpha }\right]\rangle }_1}.\\ \triangleq {\displaystyle \underset{{\lambda }_1\left(\mathsf{\alpha }\right)}{\overset{{\omega }_1\left(\mathsf{\alpha }\right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\mathsf{\alpha }\right)}{\overset{{\omega }_{TI}\left(\mathsf{\alpha }\right)}{\int }}{S}^{\left(n\right)}\left[j_n;\dots ;j_1;U^{\left(n\right)};A^{(n)};\mathsf{\alpha }\right]d{x}_1\dots d{x}_{TI} }}.\end{array}$$

(114)

The various quantities appearing in the expression presented in Equation (113) are defined as follows:

(i)

The n^th-level adjoint sensitivity function $A^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)\triangleq {\left[a^{(n)}\left(1;j_{n-1};\dots ;j_1;x\right),\dots ,a^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)\right]}^{†}$, which appears in the inner product on the right side of Equation (113), is the solution of the following n^th-Level Adjoint Sensitivity System (n^th-LASS):

$$\begin{array}{l}A{M}^{\left(n\right)}\left[2^{n-1}\times 2^{n-1};U^{\left(n\right)}\left(2^{n-1};x\right);\mathsf{\alpha }\right]A^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)\\ =Q_A^{\left(n\right)}\left[2^{n-1};j_{n-1};\dots ;j_1;U^{\left(n\right)}\left(2^{n-1};j_{n-2};\dots ;j_1;x\right);\mathsf{\alpha }\right] ,\end{array}$$

(115)

subject to boundary conditions represented in operator form as follows:

$$\begin{array}{l}{\left\{B_A^{\left(n\right)}\left[2^{n-1};U^{\left(n\right)};A^{(n)};\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}=0\left[2^{n-1}\right]; x\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right); \\ j_1=1,\dots ,TP; j_2=1,\dots ,j_1; \dots ; j_{n-1}=1,\dots ,j_{n-2}.\end{array}$$

(116)

The quantities which appear in the definition of the n^th-LASS, cf. (115) and (116), are defined as follows:

$$\begin{array}{l}{Q}_A^{\left(n\right)}\left[2^{n-1};j_{n-1};\dots ;j_1;U^{\left(n\right)};\mathsf{\alpha }\right]\triangleq \{q_A^{\left(n\right)}\left[1;j_{n-1};\dots ;j_1;U^{\left(n\right)};\mathsf{\alpha }\right],\dots \\ {\dots ,q_A^{\left(n\right)}\left[2^{n-1};j_{n-1};\dots ;j_1;U^{\left(n\right)};\mathsf{\alpha }\right]\}}^{†}, j_1=1,\dots ,TP; \dots ; j_{n-1}=1,\dots ,j_{n-2};\end{array}$$

(117)

$$q_A^{\left(n\right)}\left[1;j_{n-1};\dots ;j_1;U^{\left(n\right)}\left(x\right);\mathsf{\alpha }\right]\triangleq \partial S^{\left(n-1\right)}\left[j_{n-1};\dots ;j_1;U^{\left(n\right)};\mathsf{\alpha }\right]/\partial u\left(x\right);$$

(118)

$$q_A^{\left(n\right)}\left[2;j_{n-1};\dots ;j_1;U^{\left(n\right)}\left(x\right);\mathsf{\alpha }\right]\triangleq \partial S^{\left(n-1\right)}\left[j_{n-1};\dots ;j_1;U^{\left(n\right)};\mathsf{\alpha }\right]/\partial a^{\left(1\right)}\left(x\right);$$

(119)

$$\begin{array}{l}for n\ge 3 : q_A^{\left(n\right)}\left[2^k+i;j_{n-1};\dots ;j_1;U^{\left(n\right)}\left(x\right);\mathsf{\alpha }\right]\\ \triangleq \frac{\partial S^{\left(n-1\right)}\left[j_{n-1};\dots ;j_1;U^{\left(n\right)};\mathsf{\alpha }\right]}{\partial a^{(k+1)}\left(i;j_k;\dots ;j_1;x\right)}; k=1,\dots ,n-2; i=1,\dots ,2^k.\end{array}$$

(120)

$$U^{\left(n\right)}\left(2^{n-1};j_{n-2};\dots ;j_1;x\right)\triangleq \left(\begin{array}{c}{U}^{\left(n-1\right)}\left(2^{n-2};j_{n-3};\dots ;j_1;x\right)\\ A^{(n-1)}\left(2^{n-2};j_{n-2};\dots ;j_1;x\right)\end{array}\right);$$

(121)

$$\begin{array}{l}A{M}^{\left(n\right)}\left[2^{n-1}\times 2^{n-1};U^{\left(n\right)};\mathsf{\alpha }\right]\triangleq {\left[V{M}^{\left(n\right)}\left(2^{n-1}\times 2^{n-1};U^{\left(n\right)};\mathsf{\alpha }\right)\right]}^{*}\\ =\left(\begin{array}{cc}{\left\{{\left[V{M}^{\left(n-1\right)}\left(2^{n-2}\times 2^{n-2}\right)\right]}^{*}\right\}}^{†}& {\left\{{\left[V{M}_{21}^{\left(n\right)}\left(2^{n-2}\times 2^{n-2}\right)\right]}^{*}\right\}}^{†}\\ 0\left[2^{n-2}\times 2^{n-2}\right]& {\left\{{\left[V{M}_{22}^{\left(n\right)}\left(2^{n-2}\times 2^{n-2}\right)\right]}^{*}\right\}}^{†}\end{array}\right),\end{array}$$

(122)

The variational matrix $V{M}^{\left(n\right)}\left[2^{n-1}\times 2^{n-1};U^{\left(n\right)}\left(2^{n-1};j_{n-2};\dots ;j_1;x\right);\mathsf{\alpha }\right]$ comprises $\left(2^{n-1}\times 2^{n-1}\right)$ block matrices, each comprising $T{D}^2$ components/elements. Thus, the matrix $V{M}^{\left(n\right)}\left[2^{n-1}\times 2^{n-1};U^{\left(n\right)}\left(2^{n-1};j_{n-2};\dots ;j_1;x\right);\mathsf{\alpha }\right]$ comprises a total of $\left(2^{n-1}\times 2^{n-1}\right)T{D}^2$ components/elements. The quantity $0\left[2^{n-2}\times 2^{n-2}\right]$ denotes a block matrix having $2^{n-2}$ components, each component being a $TD\times TD$-dimensional matrix having identically zero components. The other submatrices appearing in the definition of $V{M}^{\left(n\right)}\left[2^{n-1}\times 2^{n-1};U^{\left(n\right)}\left(2^{n-1};j_{n-2};\dots ;j_1;x\right);\mathsf{\alpha }\right]$ are as follows:

$$\begin{array}{l}V{M}_{21}^{\left(n\right)}\left(2^{n-2}\times 2^{n-2};j_{n-2};\dots ;j_1;x\right)\triangleq \\ -\frac{\partial Q_A^{\left(n-1\right)}\left[2^{n-2};j_{n-2};\dots ;j_1;U^{\left(n-2\right)}\left(2^{n-2};j_{n-3};\dots ;j_1;x\right);\mathsf{\alpha }\right]}{\partial U^{\left(n-1\right)}\left(2^{n-2};j_{n-3};\dots ;j_1;x\right)}\\ +\frac{\partial \left\{A{M}^{\left(n-1\right)}\left[2^{n-2}\times 2^{n-2};U^{\left(n-1\right)};\mathsf{\alpha }\right]A^{(n-1)}\left(2^{n-2};j_{n-2};\dots ;j_1;x\right)\right\}}{\partial U^{\left(n-1\right)}\left(2^{n-2};j_{n-3};\dots ;j_1;x\right)};\end{array}$$

(123)

$$V{M}_{22}^{\left(n\right)} \left(2^{n-2}\times 2^{n-2}\right)\triangleq A{M}^{\left(n-1\right)}\left[2^{n-2}\times 2^{n-2};U^{\left(n-1\right)};\mathsf{\alpha }\right];$$

(124)

(ii)

The quantity $Q_V^{\left(n\right)}\left(2^{n-1};U^{\left(n\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right)$, which appears in the inner product on the right side of Equation (113), is defined as follows:

$$\begin{array}{l} Q_V^{\left(n\right)}\left[2^{n-1};U^{\left(n\right)}\left(2^{n-1};j_{n-2};\dots ;j_1;x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\triangleq \left(\begin{array}{c}{Q}_V^{\left(n-2\right)}\left[2^{n-2};U^{\left(n-1\right)}\left(2^{n-2};x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\\ Q_2^{\left(n\right)}\left[2^{n-2};U^{\left(n\right)}\left(2^{n-1};x\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\end{array}\right)\\ \triangleq {\left[q_V^{\left(n\right)}\left(1;U^{\left(n\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right),\dots ,q_V^{\left(n\right)}\left(2^{n-1};U^{\left(n\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right)\right]}^{†};\end{array}$$

(125)

$$q_V^{\left(n\right)}\left[i;U^{\left(n\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right]\equiv {\displaystyle \sum _{j_n=1}^{TP}{s}_V^{\left(n\right)}\left[i;j_{n-2};\dots ;j_1;U^{\left(n\right)};\mathsf{\alpha }\right]\delta {\alpha }_{j_n}}; i=1,\dots ;2^{n-1};$$

(126)

$$\begin{array}{l}{Q}_2^{\left(n\right)}\left[2^{n-2};U^{\left(n\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right]\triangleq \frac{\partial Q_A^{\left(n-1\right)}\left[2^{n-2};j_1;U^{\left(n-1\right)};\mathsf{\alpha }\right]}{\partial \mathsf{\alpha }}\partial \mathsf{\alpha }\\ -\frac{\partial \left\{A{M}^{\left(n-1\right)}\left[2^{n-2}\times 2^{n-2};U^{\left(n-1\right)};\mathsf{\alpha }\right]A^{(n-1)}\left(2^{n-2};x\right)\right\}}{\partial \mathsf{\alpha }}\partial \mathsf{\alpha };\end{array}$$

(127)

(iii)

The inner product which appears on the right side of Equation (113) is defined in a Hilbert space ${\mathrm{H}}_n\left({\mathsf{\Omega }}_x\right)$, the elements of which are block vectors of the form ${\Psi }^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)\triangleq {\left[ {\mathsf{\psi }}^{(n)}\left(1;j_{n-1};\dots ;j_1;x\right),\dots ,{\mathsf{\psi }}^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)\right]}^{†}$, comprising $2^{n-1}$ $TD$-dimensional vectors of the form ${\mathsf{\psi }}^{\left(n\right)}\left(i;x\right)\triangleq {\left[{\psi }_1^{\left(n\right)}\left(i;x\right),\dots ,{\psi }_{TD}^{\left(n\right)}\left(i;x\right)\right]}^{†}\in {\mathrm{H}}_1\left({\mathsf{\Omega }}_x\right)$, $i=1,\dots ,2^n$. The Hilbert space ${\mathrm{H}}_n\left({\mathsf{\Omega }}_x\right)$ is endowed with an inner product of two vectors ${\Psi }^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)\in {\mathrm{H}}_n\left({\mathsf{\Omega }}_x\right)$ and ${\Phi }^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)\in {\mathrm{H}}_n\left({\mathsf{\Omega }}_x\right)$, denoted as ${\langle {\Psi }^{(n)}\left(2^{n-1};x\right),{\Phi }^{(n)}\left(2^{n-1};x\right)\rangle }_n$ and defined as follows:

$${\langle {\Psi }^{(n)}\left(2^{n-1};x\right),{\Phi }^{(n)}\left(2^{n-1};x\right)\rangle }_n\triangleq \sum _{i=1}^{2^{n-1}}{\langle {\mathsf{\psi }}^{\left(n\right)}(i;x),{\mathsf{\phi }}^{\left(n\right)}(i;x)\rangle }_1$$

(128)

(iv)

The n^th-Level Adjoint Sensitivity System (n^th-LASS) for the n^th-level adjoint function $A^{(n)}\left(2^{n-1};j_{n-1};\dots ;j_1;x\right)$ is obtained by using the inner product defined in (128), as follows:

$$\begin{array}{c}{\left\{{\langle A^{(n)}\left(2^{n-1}\right),{VM}^{(n)}\left(2^{n-1}\right)\rangle }_2\right\}}_{{\mathsf{\alpha }}^0}={\left\{{\left[P^{\left(n\right)}\left(U^{\left(n\right)};A^{(n)};V^{\left(n\right)};\mathsf{\alpha }\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ +{\left\{{\langle V^{(n)}\left(2^{n-1}\right),{AM}^{(n)}[2^{n-1}\times 2^{n-1};U^{(n)};\mathsf{\alpha }]A^{(n)}(2^{n-1};x)\rangle }_n\right\}}_{{\mathsf{\alpha }}^0}\hfill \end{array}$$

(129)

where the quantity ${\left\{{\left[P^{\left(n\right)}\left(U^{\left(n\right)};A^{(n)};V^{\left(n\right)};\mathsf{\alpha }\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}$ denotes the corresponding bilinear concomitant on the domain’s boundary, evaluated at the nominal values for the parameters and respective state functions.

(v)

The quantity ${\left\{{\left[{\widehat{P}}^{\left(n\right)}\left(U^{\left(n\right)};A^{(n)};\delta \mathsf{\alpha }\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\mathsf{\alpha }}^0}$, which appears on the right side of Equation (113), denotes residual boundary terms which may have not vanished automatically after having used the variational forward and adjoint boundary conditions to eliminate from Equation (129) all unknown values of the n^th-level variational function $V^{\left(n\right)}\left(2^{n-1};j_{n-2};\dots ;j_1;x\right)$.

5. Discussion: Unified View of the n^th-CASAM-L and n^th-CASAM-N Methodologies

As indicated by the summary presentations in Section 3 and Section 4, the n^th-CASAM-L and n^th-CASAM-N methodologies are based on the same fundamental principles, as follows:

• The total sensitivity of a model response to the model parameters is defined as the Gateaux (G)-differential of the response with respect to variations in the model parameters and state functions, thereby giving rise to two terms: (i) the so-called direct effect term, which depends directly and solely on the variations in the model parameters, and (ii) the so-called indirect effect term, which depends solely on the variations in the model’s state functions.
• The straightforward evaluation of the indirect effect term would require performing as many large-scale computations (equivalent to solving the original equations) as there are model parameters, which is computationally impractical/unfeasible for realistic models involving many parameters.
• The need for performing impractically many large-scale computations is overcome by replacing the state function variations in the expression of the indirect effect term with the first-level adjoint sensitivity function, the computation of which requires a single large-scale computation for solving the 1^st-LASS, which is independent of parameter variations.
• Each subsequent sensitivity is treated as a model response, i.e., the first-order sensitivities are considered to be model responses for the computation of the second-order sensitivities using second-level adjoint sensitivity functions, and so on. The dimensions of the n^th-LASS increase linearly with the order n of the computed sensitivity.
• The sensitivities of order higher than first are computed in the ranking order of the first-order sensitivities, starting with the second-order sensitivities stemming from the largest first-order sensitivity, which are computed with the highest priority.
• Since the linear operators that underly linear systems admit adjoint operators, important model responses of linear systems can be functions of both the forward and the adjoint state functions. Such responses cannot be encountered in nonlinear problems, since nonlinear operators do not admit adjoint operators. In such cases, the n^th-CASAM-L should be applied, since it is most efficient computational methodology.
• When the model response of a linear system depends only on either the forward or adjoint state function, the n^th-CASAM-L and n^th-CASAM-N coincide. Thus, when the model response $R\left[\mathsf{\phi }\left(x\right);\mathsf{\alpha }\right]$ depends only on the forward state function $\mathsf{\phi }\left(x\right)$, the correspondence between the n^th-CASAM-L and n^th-CASAM-N is as follows: $u\left(x\right)\equiv \mathsf{\phi }\left(x\right)$ and $N\left[u\left(x\right);\mathsf{\alpha }\right]\equiv L\left(\mathsf{\alpha }\right)$. On the other hand, when the model response $R\left[\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ depends only on the adjoint state function $\mathsf{\psi }\left(x\right)$, the correspondence between the n^th-CASAM-L and n^th-CASAM-N is as follows: $u\left(x\right)\equiv \mathsf{\psi }\left(x\right)$ and $N\left[u\left(x\right);\mathsf{\alpha }\right]\equiv L^{*}\left(\mathsf{\alpha }\right)$.
• When the model response $R\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right);\mathsf{\alpha }\right]$ depends both on the forward state function $\mathsf{\phi }\left(x\right)$ and on the adjoint state function $\mathsf{\psi }\left(x\right)$, the correspondence between the n^th-CASAM-L and n^th-CASAM-N is as follows: $u\left(x\right)\triangleq {\left[\mathsf{\phi }\left(x\right),\mathsf{\psi }\left(x\right)\right]}^{†}$, $N\left[u\left(x\right);\mathsf{\alpha }\right]\triangleq N\left(\mathsf{\alpha }\right)\triangleq \left(\begin{array}{cc}L\left(\mathsf{\alpha }\right)& 0\\ 0& L^{*}\left(\mathsf{\alpha }\right)\end{array}\right)$, $Q\left(x,\mathsf{\alpha }\right)\equiv {\left[q_{\phi }\left(x;\mathsf{\alpha }\right),q_{\psi }\left(x;\mathsf{\alpha }\right)\right]}^{†}$, $B\left[u\left(x\right);\mathsf{\alpha }; x\right]\equiv {\left[B_{\phi }\left(x;\mathsf{\alpha }\right) ,B_{\psi }\left(x;\mathsf{\alpha }\right)\right]}^{†}$, and $C\left(x,\mathsf{\alpha }\right)\equiv {\left[c_{\phi }\left(x;\mathsf{\alpha }\right),c_{\psi }\left(x;\mathsf{\alpha }\right)\right]}^{†}$. If such a concatenation of the original forward and adjoint models for linear systems were implemented in practice, however, such a concatenation could lead to an unnecessary computational burden by comparison to the n^th-CASAM-L framework, which treats separately the component forward and adjoint linear systems.
• Within both the n^th-CASAM-L and n^th-CASAM-N, the sizes of the n^th-level adjoint sensitivity systems increase linearly with increasing level n, which represents the order of the sensitivities being computed. This indicates that the number of large-scale computations increases linearly with the order of sensitivities. In this sense, the n^th-CASAM-L and n^th-CASAM-N overcome the curse of dimensionality [1] in sensitivity analysis, since, in contradistinction to the n^th-CASAM-L and n^th-CASAM-N methodologies, the number of large-scale computations needed to compute the n^th-order sensitivities using conventional methods increases exponentially with the sensitivities’ order. Of course, computing sensitivities by using finite difference formulas in conjunction with re-computations using altered parameter values could be a “solution of last resort” for selected sensitivities. However, as has been shown in [29,43] for the linear and nonlinear illustrative paradigm problems reviewed in Section 3 and Section 4, respectively, using finite differences becomes not only computationally unfeasible for obtaining higher-order sensitivities because of the “curse of dimensionality,” but finding the optimal step size to minimize the error between the finite difference result and the exact result is practically impossible to achieve unless one knew beforehand what the exact result is (which would be possible only by using the CASAM-L and n^th-CASAM-N methodologies). As has been demonstrated in [29,43], a step size that is “too small” produces large errors, while a step-size which is “too large” produces even larger errors or even non-convergence. However, unless one knows ahead of time the exact result, it is not possible even to know that the finite difference formula has produced a correct or an erroneous result.
• Ongoing work aims at merging (a) concepts of artificial intelligence (AI) with (b) concepts underlying the n^th-CASAM methodologies and (c) concepts underlying the “BERRU-predictive modelling” methodology [63] into a new, potentially ground-breaking methodology aimed at “Intelligent High-Order Forward and Inverse Predictive Modelling” in order to achieve the following goals: (i) quantify the impact of ignorance, such as missing phenomena in the computational model and/or experimental information; (ii) reduce uncertainties in predicted model responses/results; (iii) improve the fidelity of computational models by reducing uncertainties in imprecisely known model parameters and incorporating missing phenomena, thus reducing the impact of prior ignorance.