Computation of High-Order Sensitivities of Model Responses to Model Parameters—II: Introducing the Second-Order Adjoint Sensitivity Analysis Methodology for Computing Response Sensitivities to Functions/Features of Parameters

Abstract

This work introduces a new methodology, which generalizes the extant second-order adjoint sensitivity analysis methodology for computing sensitivities of model responses to primary model parameters. This new methodology enables the computation, with unparalleled efficiency, of second-order sensitivities of responses to functions of uncertain model parameters, including uncertain boundaries and internal interfaces, for linear and/or nonlinear models. Such functions of primary model parameters customarily describe characteristic “features” of the system under consideration, including correlations modeling material properties, flow regimes, etc. The number of such “feature” functions is considerably smaller than the total number of primary model parameters. By enabling the computations of exact expressions of second-order sensitivities of model responses to model “features”, the number of required large-scale adjoint computations is greatly reduced. As shown in this work, obtaining the first- and second-order sensitivities to the primary model parameters from the corresponding response sensitivities to the feature functions can be performed analytically, thereby involving just the respective function/feature of parameters rather than the entire model. By replacing large-scale computations involving the model with relatively trivial computations involving just the feature functions, this new second-order adjoint sensitivity analysis methodology reaches unsurpassed efficiency. The applicability and unparalleled efficiency of this “2nd-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (2nd-FASAM) is illustrated using a paradigm particle transport model that involves feature functions of many parameters, while admitting closed-form analytic solutions. Ongoing work will generalize the mathematical framework of the 2nd-FASAM to enable the computation of arbitrarily high-order sensitivities of model responses to functions/features of model parameters.

1. Introduction

The accompanying Part I [1] has reviewed the need for computing high-order sensitivities and has highlighted their subsequent uses for performing sensitivity analyses of model responses to model parameters, quantification of uncertainties induced in responses by uncertainties in parameters, and their essential contributions to high-order predictive modeling—which combines computational with experimental information to obtain best-estimate predicted responses and calibrated model parameters with reduced predicted uncertainties.

For large-scale models involving many parameters, even the first-order sensitivities are computationally very expensive to determine accurately by conventional methods. Furthermore, the “curse of dimensionality” [2] prohibits the accurate computation of higher-order sensitivities by conventional methods. Cacuci [3] has developed the “nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (abbreviated as “nth-CASAM-L”), which overcomes the curse of dimensionality in sensitivity analysis for such systems. The “nth-CASAM-L” methodology was developed specifically for linear systems because the most important model responses produced by such systems are various Lagrangian functionals which depend simultaneously on both the forward and adjoint state functions governing the respective linear system. Since nonlinear operators do not admit adjoint operators, responses that simultaneously depend on both the forward and the adjoint state functions can occur only for linear systems. Cacuci [4] has also extended his original work [5,6] on nonlinear systems by developing the “nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (abbreviated as: nth-CASAM-N). Just like the nth-CASAM-L [3], the nth-CASAM-N [4] is formulated in linearly increasing higher-dimensional Hilbert spaces (as opposed to exponentially increasing parameter-dimensional spaces), thus overcoming the curse of dimensionality in sensitivity analyses of nonlinear systems. The nth-CASAM-L and the nth-CASAM-N methodologies enable the most efficient computation of exactly-determined expressions of arbitrarily high-order sensitivities of generic linear and/or nonlinear system responses with respect to model parameters, uncertain boundaries, and uncertain internal interfaces in the model’s phase-space. Additional details regarding applications of these high-order sensitivity analysis methodologies are provided in [7,8].

The nth-CASAM-L and the nth-CASAM-N methodologies compute the sensitivities of model responses directly with respect to the model’s primary parameters. However, many models comprise not just disparate primary model parameters but also functions of such primary model parameters. Such functions customarily appear in models in the form of correlations that describe “features” of the system under consideration, such as material properties, flow regimes, etc. Usually, the number of such “feature” functions is considerably smaller than the total number of model parameters. For example, the numerical model of the OECD/NEA reactor physics benchmark analyzed in [9] comprises 21,976 uncertain primary model parameters. However, in the large-scale numerical computation which solves the Boltzmann neutron transport equation to compute the neutron flux distribution within this benchmark, the computational model [9] does not use the primary parameters directly, but uses various functions of these parameters. High-order sensitivity analyses and uncertainty quantification investigations [9] have shown that the most important of these functions are the 180 group-averaged total macroscopic cross sections, which are functions/features of the microscopic cross sections and isotopic number densities, which in turn are uncertain primary model parameters. However, a methodology for computing exact expressions of response sensitivities to functions/features of the primary model parameters is not yet available. It is the purpose of this work to introduce, for the first time, such a methodology, which enables the most efficient computation of exactly obtained expressions of sensitivities of model responses to functions/features of model parameters. This methodology will be called the “High-Order Functions/Features Adjoint Sensitivity Analysis Methodology” (nth-FASAM).

This work is structured as follows: Section 2 presents the general mathematical modeling of a generic nonlinear system comprising functions/features of uncertain parameters and boundaries. Section 3 introduces the mathematical framework of the “First-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (1st-FASAM). Section 4 illustrates the application of the 1st-FASAM to determine the first-order sensitivities of a “dosimetry” detector response for a paradigm model of particle (neutrons and/or gamma rays) transport through a shielding material. This benchmark is representative of large-scale computations, such as those performed for the OECD/NEA reactor physics benchmark mentioned above. This paradigm particle transport model admits closed-form analytical solutions for the sensitivities of the model’s response with respect to the model’s “feature” functions, thereby highlighting the efficiency of the 1st-FASAM in obtaining exact expressions for these sensitivities and computing the respective expressions most efficiently. Comparisons with the 1st-CASAM [7] are also provided, illustrating the transition from the 1st-FASAM (which provides exact sensitivities to functions/features of parameters) to the connection with the 1st-CASAM (which provides responses sensitivities directly with respect to the primary parameters). Section 5 introduces the mathematical framework of the “Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (2nd-FASAM). It is shown that the 2nd-FASAM solves the same 2nd-Level adjoint Sensitivity System (2nd-LASS) as the 2nd-CASAM, but there are only as many (large-scale) adjoint computations as there are “features”, as opposed to as many adjoint computations as there are primary parameters, which is the case for the 2nd-CASAM. Thus, there are considerably fewer large-scale computations within the framework of the 2nd-FASAM, since the transition from the sensitivities with respect to feature functions to the sensitivities with respect to the model’s primary parameters necessitates just trivial (as opposed to large-scale) computations. Section 6 illustrates the application of the 2nd-FASAM for computing the second-order sensitivities for the dosimetry-response of the same paradigm particle transport model as was analyzed in Section 4, confirming the unmatched efficiency of the 2nd-FASAM for computing exact expressions of second-order sensitivities of model responses to both functions of parameters and to the primary parameters themselves. The discussion presented in Section 7 concludes this work, noting that ongoing research aims at developing the general framework (n^th-FASAM) for computing arbitrarily high-order sensitivities of responses to functions/features of parameters, which will thus provide the most efficient methodology for computing exact response sensitivities to model parameters.

2. Modeling of a Generic Nonlinear System Comprising Functions (“Features”) of Uncertain Parameters and Boundaries

The mathematical model that underlies the numerical evaluation of a process and/or state of a physical system comprises equations that relate the system’s independent variables and parameters to the system’s state/dependent variables. These coupled equations, which are in general nonlinear, can be represented generically in operator form as follows:

$$N\left[u\left(x\right);x;g\left(\alpha \right)\right]=Q\left[x;g\left(\alpha \right)\right]\hspace{0.33em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x\left(x\right);$$

(1)

$$B\left[u\left(x\right);g\left(\alpha \right);\hspace{0.17em}\hspace{0.17em}\lambda \left(\alpha \right);\omega \left(\alpha \right)\right]=C\left[g\left(\alpha \right);\hspace{0.17em}\hspace{0.17em}\lambda \left(\alpha \right);\omega \left(\alpha \right)\right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left[\lambda \left(\alpha \right);\omega \left(\alpha \right)\right].$$

(2)

The results computed using a mathematical model are customarily called “model responses” (or “system responses” or “objective functions” or “indices of performance”). As has been discussed in [7,8], all responses can be fundamentally analyzed in terms of the following generic integral representation:

$$R\left[u\left(x\right);f\left(\alpha \right)\right]\triangleq {\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}S\left[u\left(x\right);g\left(\alpha \right);h\left(\alpha \right);x\right]d{x}_1\dots d{x}_{TI}}},$$

(3)

where $S\left[u\left(x\right);g\left(\alpha \right);h\left(\alpha \right);x\right]$ is a suitably differentiable nonlinear function of $u\left(x\right)$ and of $\alpha$.

Without loss of generality, the quantities which appear in Equations (1)–(3) can be considered to be real-valued, and have the following meanings:

1. Matrices are denoted using capital bold letters while vectors will be denoted using either capital or lower-case bold letters. The symbol “$\triangleq$” will be used to denote “is defined as” or “is by definition equal to.” Transposition will be indicated by a dagger $\left(†\right)$ superscript. The equalities in this work are considered to hold in the weak (“distributional”) sense. Both sides of Equations (1)–(3), respectively, may contain “generalized functions/functionals”, particularly Dirac-distributions and derivatives thereof.

2. The $TP$-dimensional column-vector $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}\in {ℝ}^{TP}$ represents the “vector of primary model parameters” and has components denoted as ${\alpha }_1$,…,${\alpha }_{TP}$, where $TP$ denotes the “total number of parameters” involved in the model under consideration. These model parameters usually stem from processes that are external to the system under consideration and are seldom, if ever, known precisely. The known characteristics of the model parameters usually include their nominal (expected/mean) values and, possibly, higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis), which are usually determined from experimental data and/or processes external to the physical system under consideration. Occasionally, just the lower and the upper bounds may be known for some model parameters, expressed by inequality and/or equality constraints that delimit the ranges of the system’s parameters which are known. Without loss of generality, the imprecisely known model parameters can be considered to be real-valued scalar quantities. It is important to note that the components of the vector $\alpha$ include not only parameters that appear in Equations (1) and (2), but also include parameters that may specifically occur only in the definition of the model’s response provided in Equation (3). The nominal parameter values will be denoted as ${\alpha }^0\triangleq {\left[{\alpha }_1^0,\dots ,{\alpha }_i^0,\dots ,{\alpha }_{TP}^0\right]}^{†}$; the superscript “0” will be used throughout this monograph to denote “nominal values.”

3. The $TI$-dimensional column vector $x\triangleq {\left(x_1,\dots ,x_{TI}\right)}^{†}\in {ℝ}^{TI}$ comprises the model’s independent variables, denoted as $x_i,\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI$; the sub/superscript “TI” denotes the “total number of independent variables”. The vector $x\in {ℝ}^{TI}$ is considered to be defined on a phase-space domain denoted as ${\mathsf{\Omega }}_x\left(x\right)\triangleq \left\{{\lambda }_i\left(\alpha \right)\le x_i\le {\omega }_i\left(\alpha \right);\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI\right\}$, including the particular cases when ${\lambda }_i\left(\alpha \right)=-\infty ,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\omega }_i\left(\alpha \right)=\infty$ for some independent variables $x_i,\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI$. The domain boundary $\partial {\mathsf{\Omega }}_x\left[\lambda \left(\alpha \right);\omega \left(\alpha \right)\right]\triangleq \left\{{\lambda }_i\left(\alpha \right)\cup \hspace{0.17em}{\omega }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI\right\}$ of ${\mathsf{\Omega }}_x\left[f\left(\alpha \right)\right]$ is defined to comprise the set of all of the endpoints ${\lambda }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}{\omega }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI$. For subsequent mathematical developments, it is convenient to consider that the endpoints ${\lambda }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}{\omega }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI$ are components of column-vectors $\lambda \left(\alpha \right)\triangleq {\left[{\lambda }_1\left(\alpha \right),\dots ,{\lambda }_{TI}\left(\alpha \right)\right]}^{†}$ and $\omega \left(\alpha \right)\triangleq {\left[{\omega }_1\left(\alpha \right),\dots ,{\omega }_{TI}\left(\alpha \right)\right]}^{†}$, respectively These endpoints depend on the physical system’s geometrical dimensions, which may be imprecisely known because of manufacturing tolerances, and are considered therefore to be components of the vector $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}\in {ℝ}^{TP}$ of primary model parameters. Furthermore, the boundary-endpoints ${\lambda }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}{\omega }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI$ may also depend on the parameters that define the material properties of the respective medium. For example, in models based on diffusion theory, the boundary conditions for materials facing air/vacuum are imposed on a physics-based mathematical construct called the “extrapolated boundary” of the respective spatial domain. The “extrapolated boundary” depends both on the imprecisely known physical dimensions of the system’s materials and also on the materials’ properties, such as atomic number densities and microscopic transport cross sections. Therefore, the boundary end-points can be considered, in general, to be functions of (some of) the primary model parameters.

4. The $TD$-dimensional column vector $u\left(x\right)\triangleq {\left[u_1\left(x\right),\dots ,u_{TD}\left(x\right)\right]}^{†}$ comprises the model’s dependent variables (also called “state functions”) $u_i\left(x\right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TD$; the abbreviation “$TD$” denotes “total number of dependent variables”.

5. The vector $g\left(\alpha \right)\triangleq \left[g_1\left(\alpha \right),\dots ,g_{TG}\left(\alpha \right)\right]$ is a $TG$-dimensional vector having components $g_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TF$, which are real-valued functions of (some of) the primary model parameters $\alpha \in {ℝ}^{TP}$. Such functions customarily appear in models in the form of correlations that describe “features” of the system under consideration, such as material properties, flow regimes, etc. Usually, the number of functions $g_i\left(\alpha \right)$ is considerably smaller than the total number of model parameters, i.e., $TG\ll TP$. For example, the numerical model [9] of the OECD/NEA reactor physics benchmark comprises 21,976 uncertain primary model parameters (including microscopic cross sections and isotopic number densities) but the neutron transport equation, which is solved to determine the neutron flux distribution within the benchmark, does not use these parameters directly but instead uses “group-averaged macroscopic cross sections”, which are functions/features of the microscopic cross sections and isotopic number densities (which in turn are uncertain quantities that would be components of the vector of primary model parameters). In particular, a component $g_j\left(\alpha \right)$ may simply be one of the primary model parameters ${\alpha }_j$, i.e., $g_j\left(\alpha \right)\equiv {\alpha }_j$.

6. The $TD$-dimensional column vector $N\left[u\left(x\right);x;g\left(\alpha \right)\right]\triangleq {\left(N_1,\dots ,N_{TD}\right)}^{†}$ comprises components $N_i\left[u\left(x\right);x;g\left(\alpha \right)\right],\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TD$, which are operators (including differential, difference, integral, distributions, and/or finite or infinite matrices) acting nonlinearly (in general) on the dependent variables $u\left(x\right)$, the independent variables $x$ and on the functions $g\left(\alpha \right)$ of model parameters $\alpha$.

7. The $TD$-dimensional column vector $Q\left[u\left(x\right);x;g\left(\alpha \right)\right]\triangleq {\left(q_1,\dots ,q_{TD}\right)}^{†}$, having components $q_i\left[u\left(x\right);x;g\left(\alpha \right)\right],\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TD$, denotes inhomogeneous source terms, which usually depend nonlinearly on the uncertain parameters $\alpha$.

8. The components of $B\left[u\left(x\right);g\left(\alpha \right);\hspace{0.17em}x\right]$ are nonlinear operators, while the components of $C\left[x,g\left(\alpha \right)\right]$ represent inhomogeneous boundary sources, all defined on the boundary $\partial {\mathsf{\Omega }}_x$.

9. The integral representation of the response provided in Equation (3) can represent “averaged” as well as “point-valued” quantities in the phase-space of independent variables. For example, if $R\left[u\left(x\right);f\left(\alpha \right)\right]$ represents the computation or the measurement (which would be a “detector-response”) of a quantity of interest at a point $x_d$ in the phase-space of independent variables, then $S\left[u\left(x\right);g\left(\alpha \right);h\left(\alpha \right);x\right]$ would contain a Dirac-delta functional of the form $\delta \left(x-x_d\right)$. Responses that represent “differentials/derivatives of quantities” would contain derivatives of Dirac-delta functionals in the definition of $S\left[u\left(x\right);g\left(\alpha \right);h\left(\alpha \right);x\right]$. The vector $h\left(\alpha \right)\triangleq \left[h_1\left(\alpha \right),\dots ,h_{TH}\left(\alpha \right)\right]$, having components $h_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TH$, which appears among the arguments of the function $S\left[u\left(x\right);g\left(\alpha \right);h\left(\alpha \right);x\right]$, represents functions of primary parameters that often appear solely in the definition of the response but do not appear in the mathematical definition of the model, i.e., in Equations (1) and (2). The quantity $TH$ denotes the total number of such functions which appear exclusively in the definition of the model’s response. Evidently, the response will depend directly and/or indirectly (through the “feature”-functions) on all of the primary model parameters. This fact has been indicated in Equation (3) by using the vector-valued function $f\left(\alpha \right)$ as an argument in the definition of the response $R\left[u\left(x\right);f\left(\alpha \right)\right]$ to represent the concatenation of all of the “features” of the model and response under consideration. The vector $f\left(\alpha \right)$ is thus defined as follows:

$$f\left(\alpha \right)\triangleq {\left[g\left(\alpha \right);h\left(\alpha \right);\lambda \left(\alpha \right);\omega \left(\alpha \right)\right]}^{†}\triangleq {\left[f_1\left(\alpha \right),\dots ,f_{TF}\left(\alpha \right)\right]}^{†};\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}TF\triangleq TG+TH+2TI.$$

(4)

Solving Equations (1) and (2) at the nominal parameter values, ${\alpha }^0\triangleq {\left[{\alpha }_1^0,\dots ,{\alpha }_i^0,\dots ,{\alpha }_{TP}^0\right]}^{†}$, provides the “nominal solution” $u^0\left(x\right)$, i.e., the vectors $u^0\left(x\right)$ and ${\alpha }^0$ satisfy the following equations:

$$N\left[u^0\left(x\right);g\left({\alpha }^0\right)\right]=Q\left[x,g\left({\alpha }^0\right)\right]\hspace{0.33em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x\left(x\right),$$

(5)

$$B\left[u^0\left(x\right);g\left({\alpha }^0\right);\hspace{0.17em}\hspace{0.17em}\lambda \left({\alpha }^0\right);\omega \left({\alpha }^0\right)\right]=C\left[g\left({\alpha }^0\right);\lambda \left({\alpha }^0\right);\omega \left({\alpha }^0\right)\right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left[\lambda \left({\alpha }^0\right);\omega \left({\alpha }^0\right)\right].$$

(6)

Using the nominal parameter values ${\alpha }^0$ together with the “nominal solution” $u^0\left(x\right)$ in Equation (3) yields the nominal value of the response, namely:

$$R^0\triangleq R\left[u^0\left(x\right);f\left({\alpha }^0\right)\right]\triangleq {\displaystyle \underset{{\lambda }_1\left({\alpha }^0\right)}{\overset{{\omega }_1\left({\alpha }^0\right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left({\alpha }^0\right)}{\overset{{\omega }_{TI}\left({\alpha }^0\right)}{\int }}S\left[u^0\left(x\right);g\left({\alpha }^0\right);h\left({\alpha }^0\right);x\right]d{x}_1\dots d{x}_{TI}}}.$$

(7)

In view of Equation (7), model responses $R_k\left[f\left(\alpha \right)\right],\hspace{1em}k=1,\dots ,TR$, where $TR$ denotes the “total number of responses”, can be considered to depend directly on the functions $f\left(\alpha \right)$, and would therefore admit a Taylor-series expansion around the nominal value $f^0\triangleq f\left({\alpha }^0\right)$, having the following form:

$$R_k\left[f\left(\alpha \right)\right]=R_k\left(f^0\right)+{\displaystyle \sum _{j_1=1}^{TF}{\left\{\frac{\partial R_k\left(f\right)}{\partial f_{j_1}}\right\}}_{f^0}\delta f_{j_1}}+\frac{1}{2}{\displaystyle \sum _{j_1=1}^{TF}{\displaystyle \sum _{j_2=1}^{TF}{\left\{\frac{{\partial }^2{R}_k\left(f\right)}{\partial f_{j_1}\partial f_{j_2}}\right\}}_{f^0}}}\delta f_{j_1}\delta f_{j_2}+\dots$$

(8)

where $\delta f_j\triangleq \left[f_j\left(\alpha \right)-f_j^0\right];\hspace{0.17em}\hspace{0.17em}{f}_j^0\triangleq f_j\left({\alpha }^0\right)\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}j=1,\dots ,TF$. The “sensitivities of the model response with respect to the (feature) functions” are naturally defined as being the functional derivatives of $R_k\left[f\left(\alpha \right)\right]$ with respect to the components (“features”) $f_j\left(\alpha \right)$ of $f\left(\alpha \right)$. Since $TF\ll TP$, the computations of the functional derivatives of $R_k\left[f\left(\alpha \right)\right]$ with respect to the functions $f_j\left(\alpha \right)$, which appear in Equation (8), will be considerably less expensive computationally than the computation of the functional derivatives involved in the Taylor-series of the response with respect to the model parameters. The functional derivatives of the response with respect to the parameters can be obtained from the functional derivatives of the response with respect to the “feature” functions $f_j\left(\alpha \right)$ by simply using the chain rule, i.e.:

$${\left\{\frac{\partial R_k\left(\alpha \right)}{\partial {\alpha }_{j_1}}\right\}}_{{\alpha }^0}={\displaystyle \sum _{i_1=1}^{TF}{\left\{\frac{\partial R_k\left(f\right)}{\partial f_{i_1}}\frac{\partial f_{i_1}\left(\alpha \right)}{\partial {\alpha }_{j_1}}\right\}}_{{\alpha }^0}};\hspace{0.17em}\hspace{0.17em}{\left\{\frac{{\partial }^2{R}_k\left(\alpha \right)}{\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}}\right\}}_{{\alpha }^0}=\frac{\partial }{\partial {\alpha }_{j_2}}{\displaystyle \sum _{i_1=1}^{TF}{\left\{\frac{\partial R_k\left(f\right)}{\partial f_{i_1}}\frac{\partial f_{i_1}\left(\alpha \right)}{\partial {\alpha }_{j_1}}\right\}}_{{\alpha }^0}};$$

(9)

and so on. The evaluation/computation of the functional derivatives $\partial f_{i_1}\left(\alpha \right)/\partial {\alpha }_{j_1}$, ${\partial }^2{f}_{i_1}\left(\alpha \right)/\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}$, etc., does not require computations involving the model, and is therefore trivial by comparison to the evaluation of the functional derivatives (“sensitivities”) of the response with respect to either the functions (“features”) $f_j\left(\alpha \right)$ or the model parameters ${\alpha }_i,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TP$.

The range of validity of the Taylor-series shown in Equation (8) is defined by its radius of convergence. The accuracy—as opposed to the “validity”—of the Taylor-series in predicting the value of the response at an arbitrary point in the phase-space of model parameters depends on the order of sensitivities retained in the Taylor-expansion: the higher the respective order, the more accurate the respective response value predicted by the Taylor-series. In the particular cases when the response happens to be a polynomial function of the “feature” functions $f_j\left(\alpha \right)$, the Taylor series is actually exact.

In turn, the functions $f_i\left(\alpha \right)$ can also be formally expanded in a multivariate Taylor-series around the nominal (mean) parameter values ${\alpha }^0$, namely:

$$\begin{gathered} f_i\left(\alpha \right)=f_i\left({\alpha }^0\right)+{\displaystyle \sum _{j_1=1}^{TP}{\left\{\frac{\partial f_i\left(\alpha \right)}{\partial {\alpha }_{j_1}}\right\}}_{{\alpha }^0}\delta {\alpha }_{j_1}}+\frac{1}{2}{\displaystyle \sum _{j_1=1}^{TP}{\displaystyle \sum _{j_2=1}^{TP}{\left\{\frac{{\partial }^2{f}_i\left(\alpha \right)}{\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}}\right\}}_{{\alpha }^0}}}\delta {\alpha }_{j_1}\delta {\alpha }_{j_2} \\ \hspace{0.17em}+\frac{1}{3!}{\displaystyle \sum _{j_1=1}^{TP}{\displaystyle \sum _{j_2=1}^{TP}{\displaystyle \sum _{j_3=1}^{TP}{\left\{\frac{{\partial }^3{f}_i\left(\alpha \right)}{\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}\partial {\alpha }_{j_3}}\right\}}_{{\alpha }^0}}}}\delta {\alpha }_{j_1}\delta {\alpha }_{j_2}\delta {\alpha }_{j_3}+\dots , \end{gathered}$$

(10)

The domain of validity of the Taylor-series in Equation (10) is defined by its own radius of convergence.

In practice, the model parameters are not known exactly. Even though these parameters are not bona fide random quantities, they are formally considered to be variates that obey a multivariate probability distribution function, denoted as $p_{\alpha }\left(\alpha \right)$, which is usually unknown. The various moments of $p_{\alpha }\left(\alpha \right)$ are defined in a standard manner by using the following formal definition for the “expected (or mean) value” of a function, $u\left(\alpha \right)$, of the parameters $\alpha$, which is defined over the domain of definition of $p_{\alpha }\left(\alpha \right)$:

$$E\left[u\left(\alpha \right)\right]\triangleq {\displaystyle \int u\left(\alpha \right)p_{\alpha }\left(\alpha \right)d\alpha }.$$

(11)

In particular, the vector of expected values (denoted as ${\alpha }_j^0$) of the model parameters ${\alpha }_j$ is defined as follows:

$${\alpha }^0\triangleq {\left({\alpha }_1^0,\dots ,{\alpha }_{TP}^0\right)}^{†},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\alpha }_j^0\triangleq E\left({\alpha }_j\right),\hspace{0.17em}\hspace{0.17em}j=1,\dots ,TP.$$

(12)

The covariances, $\mathrm{cov}\left({\alpha }_i,{\alpha }_j\right)$, between two parameters, ${\alpha }_i$ and ${\alpha }_j$, are defined as follows:

$$\mathrm{cov}\left({\alpha }_i,{\alpha }_j\right)\triangleq E\left(\delta {\alpha }_i\delta {\alpha }_j\right)\triangleq {\rho }_{ij}{\sigma }_i{\sigma }_j,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i,j=1,\dots ,TP.$$

(13)

In Equation (13), the quantities ${\sigma }_i$ and ${\sigma }_j$ denote the standard deviations of ${\alpha }_i$ and ${\alpha }_j$, respectively, while ${\rho }_{ij}$ denotes the correlation between the respective parameters.

The third-order correlation, $t_{j_1{j}_2{j}_3}$, among three parameters is defined as follows:

$$t_{j_1{j}_2{j}_3}{\sigma }_{j_1}{\sigma }_{j_2}{\sigma }_{j_3}\triangleq E\left(\delta {\alpha }_{j_1}\delta {\alpha }_{j_2}\delta {\alpha }_{j_3}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1,j_2,j_3=1,\dots ,TP.$$

(14)

The fourth-order correlation $q_{j_1{j}_2{j}_3{j}_4}$ among four parameters is defined as follows:

$$q_{j_1{j}_2{j}_3{j}_4}{\sigma }_{j_1}{\sigma }_{j_2}{\sigma }_{j_3}{\sigma }_{j_4}\triangleq E\left(\delta {\alpha }_{j_1}\delta {\alpha }_{j_2}\delta {\alpha }_{j_3}\delta {\alpha }_{j_4}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1,j_2,j_3,j_4=1,\dots ,TP.$$

(15)

In terms of the moments of the parameter distribution, the moments of the distribution of the function $f_i\left(\alpha \right)$ in the phase-space of model parameters are obtained by using the Taylor-series provided in Equation (10) in conjunction with the definitions provided in Equations (12)–(15) to obtain the following expressions:

(i)

The mean (or expected value) of $f_i\left(\alpha \right)$:

$$\begin{gathered} E\left[f_i\left(\alpha \right)\right]=f_i\left({\alpha }^0\right)+\frac{1}{2}{\displaystyle \sum _{j_1=1}^{TP}{\displaystyle \sum _{j_2=1}^{TP}{\left\{\frac{{\partial }^2{f}_i\left(\alpha \right)}{\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}}\right\}}_{{\alpha }^0}{\rho }_{j_1{j}_2}}}{\sigma }_{j_1}{\sigma }_{j_2} \\ \hspace{0.17em}\hspace{0.17em}+\frac{1}{3!}{\displaystyle \sum _{j_1=1}^{TP}{\displaystyle \sum _{j_2=1}^{TP}{\displaystyle \sum _{j_3=1}^{TP}{\left\{\frac{{\partial }^3{f}_i\left(\alpha \right)}{\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}\partial {\alpha }_{j_3}}\right\}}_{{\alpha }^0}{t}_{j_1{j}_2{j}_3}}}}{\sigma }_{j_1}{\sigma }_{j_2}{\sigma }_{j_3}+\dots ,\hspace{0.17em} \end{gathered}$$

(16)

(ii)

the covariance $\mathrm{cov}\left(f_k,\hspace{0.17em}{f}_{\mathcal{l}}\right)$ of two functions $f_k\left(\alpha \right)$ and $f_{\mathcal{l}}\left(\alpha \right)$:

$$\mathrm{cov}\left(f_k,\hspace{0.17em}{f}_{\mathcal{l}}\right)\hspace{0.17em}\hspace{0.17em}={\displaystyle \sum _{j_1=1}^{TP}{\displaystyle \sum _{j_2=1}^{TP}{\left\{\frac{\partial f_k\left(\alpha \right)}{\partial {\alpha }_{j_1}}\frac{\partial f_{\mathcal{l}}\left(\alpha \right)}{\partial {\alpha }_{j_2}}\right\}}_{{\alpha }^0}{\rho }_{j_1{j}_2}}}{\sigma }_{j_1}{\sigma }_{j_2}+\dots$$

(17)

Higher-order moments of the distribution of $f\left(\alpha \right)$ in terms of the moments of the distribution of the model parameters $\alpha$ are obtained similarly.

In terms of the moments of the distribution of $f\left(\alpha \right)$, the distribution of the responses $R_k\left[f\left(\alpha \right)\right]$ is obtained by using the Taylor-series shown in Equation (8). Thus, in terms of the moments of the distribution of $f\left(\alpha \right)$, the expectation value $E\left[R_k\left(f\right)\right]$ of $R_k\left[f\left(\alpha \right)\right]$ will have the following expression:

$$E\left[R_k\left(f\right)\right]=R_k\left(f^0\right)+\frac{1}{2}{\displaystyle \sum _{j_1=1}^{TF}{\displaystyle \sum _{j_2=1}^{TF}{\left\{\frac{{\partial }^2{R}_k\left(f\right)}{\partial f_{j_1}\partial f_{j_2}}\right\}}_{f^0}}}\mathrm{cov}\left(f_{j_1},f_{j_2}\right)+\dots$$

(18)

The covariance $\mathrm{cov}\left[R_k\left(f\right),R_{\mathcal{l}}\left(f\right)\right]\hspace{0.17em}\hspace{0.17em}$ of two responses $R_k\left[f\left(\alpha \right)\right]$ and $R_{\mathcal{l}}\left(f\right)\hspace{0.17em}$ will have the following expression:

$$\mathrm{cov}\left[R_k\left(f\right),R_{\mathcal{l}}\left(f\right)\right]\hspace{0.17em}\hspace{0.17em}={\displaystyle \sum _{j_1=1}^{TP}{\displaystyle \sum _{j_2=1}^{TP}{\left\{\frac{\partial R_k\left(f\right)}{\partial f_{j_1}}\frac{\partial R_{\mathcal{l}}\left(f\right)}{\partial f_{j_2}}\right\}}_{{\alpha }^0}\mathrm{cov}\left(f_{j_1},f_{j_2}\right)}}+\dots$$

(19)

3. Introducing the 1st-FASAM: First-Order Function/Feature Adjoint Sensitivity Analysis Methodology

The “First-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (1st-FASAM), which will be developed in this section, will enable the most efficient computation of model response sensitivities to the components of the “features” function $f\left(\alpha \right)$ of model parameters. The nominal (or mean) parameter values, ${\alpha }^0$, are considered to be known, but these values will differ from the true values $\alpha$, which are unknown, by variations $\delta \alpha \triangleq {\left(\delta {\alpha }_1,\dots ,\delta {\alpha }_{TP}\right)}^{†}$, where $\delta {\alpha }_i\triangleq {\alpha }_i-{\alpha }_i^0$. The parameter variations $\delta \alpha$ will induce variations $\delta f\left(\alpha \right)\triangleq {\left[\delta f_1\left(\alpha \right),\dots ,\delta f_{TF}\left(\alpha \right)\right]}^{†}$ in the vector-valued function $f\left(\alpha \right)$ and will also induce variations $v^{\left(1\right)}\left(x\right)\triangleq {\left[\delta u_1\left(x\right),\dots ,\delta u_{TD}\left(x\right)\right]}^{†}$ around the nominal solution $u^0\left(x\right)$ in the forward state functions, since the forward state functions $u\left(x\right)$ are related to $f\left(\alpha \right)$ through Equations (1) and (2). In turn, the variations $\delta f\left(\alpha \right)$ and $v^{\left(1\right)}\left(x\right)$ will induce variations $\delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ in the system’s response $R\left[u\left(x\right);f\left(\alpha \right)\right]$.

Mathematically, the quantity $\delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ is the first-order Gateaux (G)-variation of the response $R\left[u\left(x\right);f\left(\alpha \right)\right]$, induced by arbitrary variations $\left[v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ in a neighborhood around the nominal functions and parameter values $\left[u^0\left(x\right);f^0\left(\alpha \right)\right]$, and is defined as follows:

$$\delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]\triangleq {\left\{\frac{d}{d\epsilon }R\left[u^0+\epsilon v^{\left(1\right)};f^0+\epsilon \left(\delta f\right)\right]\right\}}_{\epsilon =0},$$

(20)

where $\epsilon$ is a scalar quantity. The G-variation $\delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ is an operator defined on the same domain as $R\left[u\left(x\right);f\left(\alpha \right)\right]$ and has the same range as $R\left[u\left(x\right);f\left(\alpha \right)\right]$. The existence of the G-variation $\delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ does not guarantee its numerical computability. Numerical methods most often require that $\delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ be linear in $\left[v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ in a neighborhood $\left[u^0+\epsilon v^{\left(1\right)};f^0+\epsilon \left(\delta f\right)\right]$ around $\left[u^0\left(x\right);f^0\left(\alpha \right)\right]$, thereby admitting a total first-order G-derivative, which will be denoted as $dR\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$. The necessary and sufficient conditions for a nonlinear operator $R\left[u\left(x\right);f\left(\alpha \right)\right]$ to admit a total G-differential in a neighborhood $\left[u^0+\epsilon v^{\left(1\right)};f^0+\epsilon \left(\delta f\right)\right]$ around $\left[u^0\left(x\right);f^0\left(\alpha \right)\right]$ are as follows:

(i)

$R\left[u\left(x\right);f\left(\alpha \right)\right]$ satisfies a weak Lipschitz condition at $\left[u^0\left(x\right);f^0\left(\alpha \right)\right]$, namely:

$$‖R\left[u^0+\epsilon v^{\left(1\right)};f^0+\epsilon \left(\delta f\right)\right]-R\left(u^0;f^0\right)‖\le k‖\epsilon \left(u^0;f^0\right)‖,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k<\infty .$$

(21)

(ii)

$R\left[u\left(x\right);f\left(\alpha \right)\right]$ satisfies the following condition for a scalar $\epsilon \hspace{0.17em}$ and vectors $\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}_2, f_2$:

$$\begin{array}{l}R\left[u^0+\epsilon v^{\left(1\right)}+\hspace{0.17em}\epsilon v_2;f^0+\epsilon \left(\delta f\right)+\epsilon f_2\right]-R\left[u^0+\epsilon v^{\left(1\right)};f^0+\epsilon \left(\delta f\right)\right]\\ -R\left[u^0+\epsilon v_2;f^0+\epsilon f_2\right]+R\left(u^0;f^0\right)=o\left(\epsilon \right)\hspace{0.33em}.\end{array}$$

(22)

In practice, the relations provided in Equations (21) and (22) are seldom used directly, since the computation of the expression on the right-hand side of Equation (20) reveals immediately if the respective expression is linear (or not) in the vectors $v^{\left(1\right)}\left(x\right)$ and/or $\delta f\left(\alpha \right)$.

Numerical methods (e.g., Newton’s method and variants thereof) for solving Equations (1)and (2) also require the existence of the first-order G-derivatives of the original model equations. Therefore, the conditions provided in Equations (21) and (22) are henceforth considered to be satisfied by the model responses and also by the operators underlying the physical system modeled by Equations (1)–(3), which implies that all of the operators/functions considered in this work admit G-derivatives.

When the 1st-order G-variation $\delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]$ satisfies the conditions provided in (21) and (22), it can be written as follows:

$$\begin{gathered} \delta R\left[u^0\left(x\right);f^0\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]\triangleq {\left\{dR\left[u\left(x\right);f\left(\alpha \right);\delta f\right]\right\}}_{dir}+{\left\{dR\left[u\left(x\right);f\left(\alpha \right);v^{\left(1\right)}\left(x\right)\right]\right\}}_{ind} \\ \triangleq {\left\{\frac{d}{d\epsilon }{\displaystyle \underset{{\lambda }_1\left({\alpha }^0\right)+\epsilon \left(\delta {\lambda }_1\right)}{\overset{{\omega }_1\left({\alpha }^0\right)+\epsilon \left(\delta {\omega }_1\right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left({\alpha }^0\right)+\epsilon \left(\delta {\lambda }_{TI}\right)}{\overset{{\omega }_{TI}\left({\alpha }^0\right)+\epsilon \left(\delta {\omega }_{TI}\right)}{\int }}S\left[u^0+\epsilon v^{\left(1\right)},g^0+\epsilon \left(\delta g\right);h^0+\epsilon \left(\delta h\right);x\right]d{x}_1\dots d{x}_{TI}}}\right\}}_{\epsilon =0}. \end{gathered}$$

(23)

In Equation (23), the “direct-effect” term ${\left\{dR\left[u\left(x\right);f\left(\alpha \right);\delta f\right]\right\}}_{dir}$ comprises only dependencies on $\delta f\left(\alpha \right)$ and is defined as follows:

$${\left\{dR\left[u\left(x\right);f;\delta f\right]\right\}}_{dir}\triangleq {\left\{\frac{\partial R\left(u;f\right)}{\partial f}\right\}}_{{\alpha }^0}\delta f\hspace{0.17em}\triangleq \hspace{0.17em}{\displaystyle \sum _{j_1=1}^{TF}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);f\left(\alpha \right)\right]\right\}}_{dir}\delta f_{j_1}},$$

(24)

where:

$$\frac{\partial \left[\right]}{\partial f}\delta f\triangleq {\displaystyle \sum _{i=1}^{TF}\frac{\partial \left[\right]}{\partial f_i}\delta f_i}={\displaystyle \sum _{i=1}^{TG}\frac{\partial \left[\right]}{\partial g_i}\delta g_i}+{\displaystyle \sum _{i=1}^{TH}\frac{\partial \left[\right]}{\partial h_i}\delta h_i}+{\displaystyle \sum _{i=1}^{TI}\frac{\partial \left[\right]}{\partial {\lambda }_i}\delta {\lambda }_i}+{\displaystyle \sum _{i=1}^{TI}\frac{\partial \left[\right]}{\partial {\omega }_i}\delta {\omega }_i},$$

(25)

$$\begin{array}{l}For\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TG:\\ {\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);f\left(\alpha \right)\right]\right\}}_{dir}\delta f_{j_1}\triangleq {\left\{{\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}\frac{\partial S\left(u;g;h\right)}{\partial g_i}d{x}_1\dots d{x}_{TI}}}\right\}}_{{\alpha }^0}\delta g_i;\end{array}$$

(26)

$$\begin{array}{l}For\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=TG+1,\dots ,TG+TH:\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);f\left(\alpha \right)\right]\right\}}_{dir}\delta f_{j_1}\\ \triangleq {\left\{{\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}\frac{\partial S\left(u;h\right)}{\partial h_i}d{x}_1\dots d{x}_{TI}}}\right\}}_{{\alpha }^0}\delta h_i,\hspace{0.17em}\hspace{0.17em}where\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=j_1-TG\hspace{0.17em};\end{array}$$

(27)

$$\begin{array}{l}For\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}{j}_1=TG+TH+1,\dots ,TG+TH+TI:\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);f\left(\alpha \right)\right]\right\}}_{dir}\delta f_{j_1}\\ \triangleq {\left\{{\displaystyle \underset{{\lambda }_1}{\overset{{\omega }_1}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{i-1}}{\overset{{\omega }_{i-1}}{\int }}d{x}_{i-1}{\displaystyle \underset{{\lambda }_{1+1}}{\overset{{\omega }_{i+1}}{\int }}d{x}_{i+1}\dots }}{\displaystyle \underset{{\lambda }_{TI}}{\overset{{\omega }_{TI}}{\int }}d{x}_{TI}S\left[u\left(x_1\hspace{1em}\hspace{0.17em},.,{\omega }_i\left(\alpha \right),.,x_{N_x}\right);g;h\right]}}\right\}}_{{\alpha }^0}\delta {\omega }_i\left(\alpha \right)\hspace{0.17em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\\ where\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=j_1-TG\hspace{0.17em}-TH;\end{array}$$

(28)

$$\begin{array}{l}For\hspace{1em}{j}_1=TG+TH+TI+1,\dots ,TG+TH+2TI:\hspace{1em}\hspace{1em}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);f\left(\alpha \right)\right]\right\}}_{dir}\delta f_{j_1}\triangleq \hspace{0.17em}\\ -{\left\{{\displaystyle \underset{{\lambda }_1}{\overset{{\omega }_1}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{i-1}}{\overset{{\omega }_{i-1}}{\int }}d{x}_{i-1}{\displaystyle \underset{{\lambda }_{1+1}}{\overset{{\omega }_{i+1}}{\int }}d{x}_{i+1}}}\dots {\displaystyle \underset{{\lambda }_{TI}}{\overset{{\omega }_{TI}}{\int }}d{x}_{TI}S\left[u\left(x_1\hspace{1em}\hspace{0.17em},.,{\lambda }_i\left(\alpha \right),.,x_{N_x}\right);g;h\right]}}\right\}}_{{\alpha }^0}\delta {\lambda }_i,\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}where\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=j_1-TG\hspace{0.17em}-TH-TI.\end{array}$$

(29)

The notation on the left-hand side of Equation (25) represents an inner product between two vectors (so it comprises an implied multiplication of a vector with a transposed vector), but the “dagger” ($†$) which indicates “transposition” has been omitted, in order to keep the notation as simple as possible. “Daggers” indicating transposition will also be omitted in other inner products, whenever possible, while avoiding ambiguities.

The direct-effect term can be computed after having solved Equations (1) and (2) to obtain the nominal values, $u^0\left(x\right)$, of the dependent variables. On the other hand, the quantity ${\left\{dR\left[u\left(x\right);f\left(\alpha \right);v^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}$ defined in Equation (23) comprises only variations in the state functions—and is therefore called the “indirect-effect term”—and is defined as follows:

$${\left\{dR\left[u\left(x\right);f\left(\alpha \right);v^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}\triangleq {\left\{{\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}d{x}_{TI}\frac{\partial S\left(u;g;h\right)}{\partial u}{v}^{\left(1\right)}\left(x\right)}}\right\}}_{{\alpha }^0},$$

(30)

where

$$\frac{\partial \left[\right]}{\partial u}{v}^{\left(1\right)}\left(x\right)\triangleq {\displaystyle \sum _{i=1}^{TD}\frac{\partial \left[\right]}{\partial u_i\left(x\right)}\delta u_i\left(x\right)}.$$

(31)

The “indirect-effect” term induces variations in the response through the variations in the state functions, which are, in turn, caused by the parameter variations through the equations underlying the model. The indirect-effect term can be quantified only after having determined the variations $v^{\left(1\right)}\left(x\right)$ in terms of the variations $\delta g$, $\delta \lambda$, and $\delta \omega$.

The first-order relationship between the vectors $v^{\left(1\right)}\left(x\right)$ and the variations $\delta g$, $\delta \lambda$, and $\delta \omega$ is determined by solving the equations obtained by applying the definition of the G-differential to Equations (1) and (2), which yields the following equations:

$${\left\{\frac{d}{d\epsilon }N\left[u^0+\epsilon v^{\left(1\right)}\left(x\right);g\left({\alpha }^0\right)+\epsilon \left(\delta g\right)\right]\right\}}_{\epsilon =0}={\left\{\frac{d}{d\epsilon }Q\left[x,g\left({\alpha }^0\right)+\epsilon \left(\delta g\right)\right]\right\}}_{\epsilon =0},$$

(32)

$$\begin{array}{l}{\left\{\frac{d}{d\epsilon }B\left[u^0+\epsilon v^{\left(1\right)}\left(x\right);g\left({\alpha }^0\right)+\epsilon \left(\delta g\right);\hspace{0.17em}\hspace{0.17em}\lambda \left({\alpha }^0\right)+\epsilon \left(\delta \lambda \right);\omega \left({\alpha }^0\right)+\epsilon \left(\delta \omega \right)\right]\right\}}_{\epsilon =0}\\ -{\left\{\frac{d}{d\epsilon }C\left[\hspace{0.17em}g\left({\alpha }^0\right)+\epsilon \left(\delta g\right);\hspace{0.17em}\hspace{0.17em}\lambda \left({\alpha }^0\right)+\epsilon \left(\delta \lambda \right);\omega \left({\alpha }^0\right)+\epsilon \left(\delta \omega \right)\right]\right\}}_{\epsilon =0}=0\hspace{0.17em},\end{array}$$

(33)

Carrying out the differentiations with respect to $\epsilon$ in Equations (32) and (33), and setting $\epsilon =0$ in the resulting expressions yields the following equations:

$${\left\{N^{\left(1\right)}\left(u;g\right)v^{\left(1\right)}\left(x\right)\right\}}_{{\alpha }^0}={\left\{q_V^{\left(1\right)}\left(u;g;\delta g\right)\right\}}_{{\alpha }^0},\hspace{1em}x\in {\mathsf{\Omega }}_x,$$

(34)

$${\left\{b_V^{\left(1\right)}\left(u;g;\lambda ;\omega ;v^{\left(1\right)}\hspace{0.17em};\delta g;\delta \lambda ;\delta \omega \right)\right\}}_{{\alpha }^0}=0,\hspace{1em}x\in \partial {\mathsf{\Omega }}_x.$$

(35)

In Equations (34) and (35), the superscript “(1)” indicates “1st-Level” and the various quantities which appear in these equations are defined as follows:

$$N^{\left(1\right)}\left(u;g\right)\triangleq \left\{\frac{\partial N\left(\hspace{0.17em}u;g\right)}{\partial u}\right\}\triangleq {\left\{\frac{\partial N_i}{\partial u_j}\right\}}_{TD\times TD};$$

(36)

$$q_V^{\left(1\right)}\left(u;g;\delta g\right)\triangleq \frac{\partial \left[Q\left(g\right)-N\left(\hspace{0.17em}u;g\right)\right]}{\partial g}\hspace{0.17em}\delta g\triangleq \hspace{0.17em}{\displaystyle \sum _{j_1=1}^{TG}{s}_V^{\left(1\right)}\left(j_1;u;g\right)\delta g_{j_1}};$$

(37)

$$\begin{array}{l}{\left\{b_V^{\left(1\right)}\left(u;g;\lambda ;\omega ;v^{\left(1\right)};\delta g;\delta \lambda ;\delta \omega \right)\right\}}_{{\alpha }^0}\triangleq {\left\{\frac{\partial B\left(u;g;\lambda ;\omega \right)}{\partial u}\right\}}_{{\alpha }^0}{v}^{\left(1\right)}+\hspace{0.17em}\left\{\frac{\partial \left[B\left(u;g;\lambda ;\omega \right)-C\left(g;\lambda ;\omega \right)\right]}{\partial g}\delta g\right.\\ \hspace{0.17em}+\frac{\partial \left[B\left(u;g;\lambda ;\omega \right)-C\left(g;\lambda ;\omega \right)\right]}{\partial \lambda }\delta \lambda {\left.+\frac{\partial \left[B\left(u;g;\lambda ;\omega \right)-C\left(g;\lambda ;\omega \right)\right]}{\partial \omega }\delta \omega \right\}}_{{\alpha }^0}.\end{array}$$

(38)

The system of equations comprising Equations (34) and (35) is called the “1st-Level Variational Sensitivity System” (1st-LVSS). The solution, $v^{\left(1\right)}\left(x\right)$, of the 1st-LVSS will be a function of the variations $\delta g$, $\delta \lambda$, and $\delta \omega$. Hence, when the function $v^{\left(1\right)}\left(x\right)$ is introduced into the expression of the indirect-effect term defined in Equation (30), it will introduce dependencies of the response sensitivities on $\delta g$, $\delta \lambda$, and $\delta \omega$, which will be in addition to the dependencies displayed by the direct-effect term defined in Equation (24). As Equations (34) and (35) indicate, every parameter variation $\delta {\alpha }_{j_1}$, $j_1=1,\dots ,TP$, will induce, directly or indirectly, a variations in the model’s state variables. In principle, therefore, for every parameter variation $\delta {\alpha }_{j_1}$, $j_1=1,\dots ,TP$, there would be a corresponding solution $v^{\left(1\right)}\left(j_1;x\right)$, $j_1=1,\dots ,TP$, of the 1st-LVSS. Thus, if the effect of every parameter variation were of interest, then the 1st-LVSS would need to be solved $TP$ times, with distinct right-hand sides and boundary conditions for each parameter variation $\delta {\alpha }_{j_1}$, which would require at least $TP$ large-scale computations. If only variations induced by the functions $\delta g$, $\delta \lambda$, and/or $\delta \omega$ were of interest, then fewer than $TP$ large-scale computations would be required.

However, solving the 1st-LVSS can be avoided altogether by using the ideas underlying the “adjoint sensitivity analysis methodology” as originally conceived by Cacuci [5,6] and subsequently generalized by Cacuci [7,8] to enable the computation of arbitrarily high-order response sensitivities to model parameters. Thus, the need for computing the vectors $v^{\left(1\right)}\left(j_1;x\right)$, $j_1=1,\dots ,TP$ is eliminated by expressing the indirect-effect term defined in Equation (30) in terms of the solutions of the “1st-Level Adjoint Sensitivity System” (1st-LASS), which is constructed by introducing a (real) Hilbert space denoted as ${\mathsf{H}}_1\left({\mathsf{\Omega }}_x\right)$, endowed with an inner product of two vectors $w^{(1)}\left(x\right)\in {\mathsf{H}}_1$ and $w^{(2)}\left(x\right)\in {\mathsf{H}}_1$ which is denoted as ${〈w^{(1)},w^{(2)}〉}_1$ and defined as follows:

$${〈w^{(1)},w^{(2)}〉}_1\triangleq \hspace{0.17em}{\left\{{\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}\left[w^{(1)}\left(x\right)·w^{(2)}\left(x\right)\right]d{x}_1\dots d{x}_{TI}}}\right\}}_{{\alpha }^0}.$$

(39)

In Equation (39), the “dagger” ($†$), which indicates “transposition,” has been omitted to simplify the notation for the scalar product $w^{(1)}\left(x\right)·w^{(1)}\left(x\right)\hspace{0.17em}\hspace{0.17em}\triangleq {\displaystyle \sum _{i=1}^{TD}{w}_i^{(1)}\left(x\right)w_i^{(2)}}\left(x\right)$.

The 1st-LASS is now constructed by considering a vector $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$, which is an element in ${\mathsf{H}}_1\left({\mathsf{\Omega }}_x\right)$ but is otherwise arbitrary at this stage, and by using Equation (39) to form the inner product of $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$ with the relation provided in Equation (34) to obtain:

$${\left\{{〈a^{(1)},\hspace{0.17em}{N}^{\left(1\right)}\left(u;g\right)v^{\left(1\right)}〉}_1\right\}}_{{\alpha }^0}={\left\{{〈a^{(1)},\hspace{0.17em}{q}_V^{\left(1\right)}\left(u;g;\delta g\right)〉}_1\right\}}_{{\alpha }^0},\hspace{1em}x\in {\mathsf{\Omega }}_x.$$

(40)

The left-hand side of Equation (40) is transformed by using the definition of the adjoint operator in ${\mathsf{H}}_1\left({\mathsf{\Omega }}_x\right)$, as follows:

$${\left\{{〈a^{(1)},\hspace{0.17em}{N}^{\left(1\right)}\left(u;g\right)v^{\left(1\right)}〉}_1\right\}}_{{\alpha }^0}={\left\{{〈A^{\left(1\right)}\left(u;g\right)\hspace{0.17em}{a}^{(1)},\hspace{0.17em}{v}^{\left(1\right)}〉}_1\right\}}_{{\alpha }^0}+{\left\{{\left[P^{\left(1\right)}\left(u;g;\lambda ;\omega ;v^{\left(1\right)};\hspace{0.17em}{a}^{(1)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0},$$

(41)

where ${\left[P^{\left(1\right)}\left(u;g;\hspace{0.17em}\lambda ;\omega ;a^{(1)};v^{\left(1\right)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}$ denotes the associated bilinear concomitant evaluated on the domain’s boundary $\partial {\mathsf{\Omega }}_x$ and where $A^{\left(1\right)}\left(u;g\right)\triangleq {\left[N^{\left(1\right)}\left(u;g\right)\right]}^{*}$ denotes the operator formally adjoint to $N^{\left(1\right)}\left(u;g\right)$. The symbol ${\left[\right]}^{\ast }$ indicates “formal adjoint” operator.

The first term on the right-hand side of Equation (41) is now required to represent the indirect-effect term defined in Equation (30) by imposing the following relationship:

$${\left\{A^{\left(1\right)}\left(u;g\right)a^{\left(1\right)}\left(x\right)\hspace{0.17em}\right\}}_{{\alpha }^0}={\left\{\partial S\left(u;g;h\hspace{0.17em}\right)/\partial u\right\}}_{{\alpha }^0}\triangleq q_A^{\left(1\right)}\left[u\left(x\right);g;h\right],\hspace{1em}x\in {\mathsf{\Omega }}_x.$$

(42)

The domain of $A^{\left(1\right)}\left(u;g\right)$ is determined by selecting the appropriate adjoint boundary and/or initial conditions, which will be denoted in operator form as:

$${\left\{b_A^{\left(1\right)}\left(u;a^{\left(1\right)};g\right)\right\}}_{{\alpha }^0}=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x.$$

(43)

The above boundary conditions for the adjoint operator $A^{\left(1\right)}\left(u;g\right)$ are obtained by imposing the following requirements: (i) they must be independent of unknown values of $v^{\left(1\right)}\left(x\right)$ and $\delta g$; and (ii) the substitution of the boundary and initial conditions represented by Equations (35) and (43) into the expression of ${\left\{{\left[P^{\left(1\right)}\left(u;g;\hspace{0.17em}{a}^{(1)};v^{\left(1\right)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}$ must cause all terms containing unknown values of $v^{\left(1\right)}\left(x\right)$ to vanish. Using the adjoint and forward variational boundary conditions represented by Equations (43) and (35) into (41) reduces the bilinear concomitant ${\left\{{\left[P^{\left(1\right)}\left(u;g;\hspace{0.17em}{a}^{(1)};v^{\left(1\right)}\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}$ to a residual term denoted as ${\left\{{\left[{\widehat{P}}^{\left(1\right)}\left(u;g;\lambda ;\hspace{0.17em}\omega ;a^{(1)};\delta g;\delta \lambda ;\delta \omega \right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}$, which will contain boundary terms involving only known values of $\delta g$, $\delta \lambda ;\delta \omega$;$g$, $u$, and $a^{(1)}$ The residual term ${\left\{{\left[{\widehat{P}}^{\left(1\right)}\left(u;g;\lambda ;\hspace{0.17em}\omega ;a^{(1)};\delta g;\delta \lambda ;\delta \omega \right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}$ is linear in $\delta g$, $\delta \lambda ;\delta \omega$, and can therefore be expressed in the following form:

$$\begin{array}{l}{\left\{{\left[{\widehat{P}}^{\left(1\right)}\left(u;g;\lambda ;\hspace{0.17em}\omega ;a^{(1)};\delta g;\delta \lambda ;\delta \omega \right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}={\left\{{\displaystyle \sum _{j_1=1}^{TG}\left[\partial {\widehat{P}}^{\left(1\right)}/\partial g_{j_1}\right]\delta g_{j_1}}\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \sum _{j_1=1}^{TI}\left[\partial {\widehat{P}}^{\left(1\right)}/\partial {\lambda }_{j_1}\right]\delta {\lambda }_{j_1}}\right\}}_{{\alpha }^0}+{\left\{{\displaystyle \sum _{j_1=1}^{TI}\left[\partial {\widehat{P}}^{\left(1\right)}/\partial {\omega }_{j_1}\right]\delta {\omega }_{j_1}}\right\}}_{{\alpha }^0}.\end{array}$$

(44)

The results obtained in (41) and (42) are now replaced in (30) to obtain the following expression of the indirect-effect term as a function of $a^{(1)}\left(x\right)$:

$$\begin{array}{l}{\left\{\delta R\left[u\left(x\right);f\left(\alpha \right);v^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}={\left\{{〈a^{(1)},\hspace{0.17em}{q}_V^{\left(1\right)}\left(u;g;\delta g\right)〉}_1\right\}}_{{\alpha }^0}\\ -{\left\{{\left[{\widehat{P}}^{\left(1\right)}\left(u;g;\lambda ;\hspace{0.17em}\omega ;a^{(1)};\delta g;\delta \lambda ;\delta \omega \right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}.\end{array}$$

(45)

Replacing in Equation (24) the result obtained in Equation (45), together with the expression for the direct-effect term provided in Equation (26), yields the following expression for the first-order G-differential of the response $R\left[u\left(x\right);f\left(\alpha \right)\right]$:

$$\begin{array}{l}{\left\{dR\left[u\left(x\right);f\left(\alpha \right);v^{\left(1\right)}\left(x\right);\delta f\left(\alpha \right)\right]\right\}}_{{\alpha }^0}={\left\{dR\left[u\left(x\right);f\left(\alpha \right);\delta f\right]\right\}}_{dir}\\ +{\left\{{〈a^{(1)},\hspace{0.17em}{q}_V^{\left(1\right)}\left(u;g;\delta g\right)〉}_1\right\}}_{{\alpha }^0}-{\left\{{\left[{\widehat{P}}^{\left(1\right)}\left(u;g;\lambda ;\hspace{0.17em}\omega ;a^{(1)};\delta g;\delta \lambda ;\delta \omega \right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}\\ \triangleq {\left\{{\displaystyle \sum _{j_1=1}^{TF}{R}^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]\delta f_{j_1}}\right\}}_{{\alpha }^0},\end{array}$$

(46)

where $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]\triangleq \partial R\left[u\left(x\right);f\left(\alpha \right)\right]/\partial f_{j_1}$ denotes the first-order sensitivity of the response $R\left[u\left(x\right);f\left(\alpha \right)\right]$ with respect to the components $f_{j_1}$ of the “feature” function $f\left(\alpha \right)\triangleq {\left[g\left(\alpha \right);h\left(\alpha \right);\lambda \left(\alpha \right);\omega \left(\alpha \right)\right]}^{†}$. Each sensitivity $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]$ is obtained by identifying the expression that multiplies the corresponding component of the variations $\delta g$, $\delta h$, $\delta \lambda$, and $\delta \omega$, respectively, on the right-hand side of Equation (46). Each of the 1st-order sensitivities $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]$ of the response $R\left[u\left(x\right);f\left(\alpha \right)\right]$ with respect to the components of the functions $g$, $h$, $\lambda$, and $\omega$ can be computed inexpensively after having obtained the 1st-level adjoint function $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$, using just quadrature formulas to evaluate the inner products involving $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$ in Equation (46). The function $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$ is obtained by solving numerically (42) and (43), which is the only large-scale computation needed for obtaining all of the first-order sensitivities.

Equations (42) and (43) are called the 1st-Level Adjoint Sensitivity System (1st-LASS). The solution, $a^{(1)}\left(x\right)\in {\mathsf{H}}_1\left({\mathsf{\Omega }}_x\right)$, of the 1st-LASS is called the 1st-level adjoint function. It is very important to note that the 1st-LASS is independent of all parameter variations $\delta g$, $\delta h$, $\delta \lambda$, and $\delta \omega$ (or, equivalently, of all parameter variations $\delta {\alpha }_{j_1}$,$j_1=1,\dots ,TP$) and therefore needs to be solved only once, regardless of the number of model parameters under consideration. Furthermore, since the 1st-LASS is linear in $a^{(1)}\left(x\right)$, solving it requires less computational effort than solving the original model, which is nonlinear in $u\left(x\right)$. The first-order sensitivity $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]$ can be represented formally in the following integral form:

$$R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]\triangleq {\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}{S}^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]d{x}_1\dots d{x}_{TI}}};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TF.$$

(47)

The functions $S^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]$ will be subsequently used for determining the exact expressions of the second-order sensitivities of the response with respect to the components of the function $f\left(\alpha \right)$ of model parameters.

3.1. Comparison between 1st-FASAM and 1st-CASAM

The general methodology for computing most efficiently the exactly-derived expressions of the first-order sensitivities of a model response directly with respect to the model’s parameters is the “First-Order Comprehensive Adjoint Sensitivity Analysis Methodology” (1st-CASAM) developed by Cacuci [5,6,7,8], which was briefly reviewed in the accompanying Part I [1]. The conceptual difference between the 1st-CASAM and the 1st-FASAM presented in the first work stem from the fact that within the 1st-CASAM the possible functions (“features”) of model parameters are not explicitly considered, so that all differentials are computed directly with respect to the basic model parameters. Consequently, the main distinctions between the 1st-CASAM and the 1st-FASAM are as follows:

(i)

Instead of the expressions provided in Equations (24)–(29), the “direct-effect” term in the 1st-CASAM, denoted as ${\left\{\delta R\left[u\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{dir}$, comprises the direct dependencies on $\delta \alpha$ and is defined as follows:

$${\left\{\delta R\left[u\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{dir}\triangleq {\left\{\frac{\partial R\left(u;\alpha \right)}{\partial \alpha }\right\}}_{{\alpha }^0}\delta \alpha \triangleq \hspace{0.17em}{\displaystyle \sum _{j_1=1}^{TP}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);\alpha \right]\right\}}_{dir}\delta {\alpha }_{j_1}},$$

(48)

where $\partial R\left(u;\alpha \right)/\partial \alpha$ denotes the partial G-derivatives of $R\left(e\right)$ with respect to $\alpha$, evaluated at the nominal parameter values, and where the following definitions were used:

$$\frac{\partial \left[\right]}{\partial \alpha }\delta \alpha \triangleq {\displaystyle \sum _{i=1}^{TP}\frac{\partial \left[\right]}{\partial {\alpha }_i}\delta {\alpha }_i},$$

(49)

$$\begin{array}{l}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);\alpha \right]\right\}}_{dir}\triangleq {\left\{{\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}\frac{\partial S\left(u;\alpha ;\alpha \right)}{\partial {\alpha }_{j_1}}d{x}_1\dots d{x}_{TI}}}\right\}}_{{\alpha }^0}\\ +{\displaystyle \sum _{j=1}^{TI}{\left\{\frac{\partial {\omega }_j\left(\alpha \right)}{\partial {\alpha }_{j_1}}{\displaystyle \underset{{\lambda }_1}{\overset{{\omega }_1}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{j-1}}{\overset{{\omega }_{j-1}}{\int }}d{x}_{i-1}{\displaystyle \underset{{\lambda }_{j+1}}{\overset{{\omega }_{j+1}}{\int }}d{x}_{i+1}\dots {\displaystyle \underset{{\lambda }_{TI}}{\overset{{\omega }_{TI}}{\int }}d{x}_{TI}S\left[u\left(x_1\hspace{1em}\hspace{0.17em},.,{\omega }_j\left(\alpha \right),.,x_{N_x}\right);\alpha \right]}}}}\right\}}_{{\alpha }^0}}\hspace{0.17em}\\ -{\displaystyle \sum _{j=1}^{TI}{\left\{\frac{\partial {\lambda }_j\left(\alpha \right)}{\partial {\alpha }_{j_1}}{\displaystyle \underset{{\lambda }_1}{\overset{{\omega }_1}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{j-1}}{\overset{{\omega }_{j-1}}{\int }}d{x}_{i-1}{\displaystyle \underset{{\lambda }_{j+1}}{\overset{{\omega }_{j+1}}{\int }}d{x}_{i+1}\dots {\displaystyle \underset{{\lambda }_{TI}}{\overset{{\omega }_{TI}}{\int }}d{x}_{TI}S\left[u\left(x_1\hspace{1em}\hspace{0.17em},.,{\lambda }_j\left(\alpha \right),.,x_{N_x}\right);\alpha \right]}}}}\right\}}_{{\alpha }^0}}.\end{array}$$

(50)

(ii)

Instead of Equations (34) and (35), the 1st-LVSS within the 1st-CASAM framework has the following expression:

$${\left\{N^{\left(1\right)}\left(u;\alpha \right)v^{\left(1\right)}\left(j_1;x\right)\right\}}_{{\alpha }^0}={\left\{s_V^{\left(1\right)}\left(j_1;u;\alpha \right)\right\}}_{{\alpha }^0},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x,$$

(51)

$${\left\{b_V^{\left(1\right)}\left[u;\alpha ;v^{\left(1\right)}\left(j_1;x\right)\right]\right\}}_{{\alpha }^0}=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right).$$

(52)

where:

$$s_V^{\left(1\right)}\left(j_1;u;\alpha \right)\triangleq \frac{\partial \left[Q\left(\alpha \right)-N\left(\hspace{0.17em}u;\alpha \right)\right]}{\partial {\alpha }_{j_1}};$$

(53)

$${\left\{b_V^{\left(1\right)}\left[u;\alpha ;v^{\left(1\right)}\left(j_1;x\right)\right]\right\}}_{{\alpha }^0}\triangleq {\left\{\frac{\partial B\left(u;\alpha \right)}{\partial u}{v}^{\left(1\right)}\left(j_1;x\right)+\frac{\partial \left[B\left(u;\alpha \right)-C\left(\alpha \right)\right]}{\partial {\alpha }_{j_1}}\right\}}_{{\alpha }^0}.$$

(54)

To determine the solutions $v^{\left(1\right)}\left(j_1;x\right)$ of the 1st-LVSS that would correspond to every parameter variation $\delta {\alpha }_{j_1}$, $j_1=1,\dots ,TP$, the 1st-LVSS would need to be solved $TP$ times, with distinct right-hand sides for each $\delta {\alpha }_{j_1}$, thus requiring $TP$ large-scale computations.

(iii)

The first-order G-differential of the response induced by variations in the model parameters and, consequently, the first-order sensitivities $\partial R\left[u\left(x\right);\alpha \right]/\partial {\alpha }_{j_1}$ of the response with respect to the model parameters have the following expressions:

$$\begin{array}{l}{\left\{\delta R\left[u\left(x\right);\alpha ;v^{\left(1\right)}\left(x\right);\delta \alpha \right]\right\}}_{{\alpha }^0}={\left\{\delta R\left[u\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{dir}+{\left\{{〈a^{(1)},\hspace{0.17em}{q}_V^{\left(1\right)}\left(u;\alpha ;\delta \alpha \right)〉}_1\right\}}_{{\alpha }^0}\\ -{\left\{{\left[{\widehat{P}}^{\left(1\right)}\left(u;\alpha ;\hspace{0.17em}{a}^{(1)};\delta \alpha \right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}\triangleq \hspace{0.17em}{\displaystyle \sum _{j_1=1}^{TP}{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);a^{(1)}\left(x\right);\alpha \right]\right\}}_{{\alpha }^0}\delta {\alpha }_{j_1}},\end{array}$$

(55)

where, for each $j_1=1,\dots ,TP$, the quantity $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]\triangleq \partial R\left[u\left(x\right);\alpha \right]/\partial {\alpha }_{j_1}$ denotes the 1st-order sensitivities of the response $R\left[u\left(x\right);\alpha \right]$ with respect to the model parameters ${\alpha }_{j_1}$ and has the following expression, for $j_1=1,\dots ,TP$:

$$\begin{array}{l}{R}^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]=-\frac{\partial {\left[{\widehat{P}}^{\left(1\right)}\left(u;a^{(1)};\alpha ;\right)\right]}_{\partial {\Omega }_x}}{\partial {\alpha }_{j_1}}\\ +{\left\{R^{\left(1\right)}\left[j_1;u\left(x\right);\alpha \right]\right\}}_{dir}+{\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}{a}^{(1)}\left(x\right)\frac{\partial \left[Q\left(\alpha \right)-N\left(\hspace{0.17em}u;\alpha \right)\right]}{\partial {\alpha }_{j_1}}d{x}_{TI}}}\\ \triangleq {\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}{S}^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]d{x}_1\dots d{x}_{TI}\triangleq \hspace{0.17em}\frac{\partial R\left[u\left(x\right);\alpha \right]}{\partial {\alpha }_{j_1}}}}.\end{array}$$

(56)

Comparing Equation (56) to Equation (46) indicates that the same first-level adjoint function $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$ is used in all of these expressions. Therefore, the 1st-LASS needs to be solved only once, to obtain $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$, regardless of whether the response sensitivities are computed directly with respect to the model parameters or whether the sensitivities are first computed with respect to the components (“features”) of the function $f\left(\alpha \right)$ and subsequently computed with respect to the components of the vector $\alpha$ of model parameters. This is as expected, since the 1st-LASS is independent of parameter variations $\delta {\alpha }_{j_1}$,$j_1=1,\dots ,TP$, and therefore needs to be solved only once, regardless of the number of “features” or model parameters under consideration.

It is more advantageous to use the 1st-FASAM to compute the sensitivities of the response to the “feature” functions provided in Equation (46), since these expressions involve fewer integrals (“quadrature”) over the 1st-level adjoint function $a^{(1)}\left(x\right)\in {\mathsf{H}}_1$ than using Equation (56), since $TG<TP$. The transition from the sensitivities with respect to “features” provided in Equation (46) to the sensitivities with respect to the model parameters provided in Equation (56) can be carried out very inexpensively by using the “chain-rule” of differentiation of compound functions, namely $\delta g_i\left(\alpha \right)=\left[\partial g_i\left(\alpha \right)/\partial \alpha \right]\delta \alpha$, $\delta h_i\left(\alpha \right)=\left[\partial h_i\left(\alpha \right)/\partial \alpha \right]\delta \alpha$, $\delta {\lambda }_i\left(\alpha \right)=\left[\partial {\lambda }_i\left(\alpha \right)/\partial \alpha \right]\delta \alpha$, $\delta {\omega }_i\left(\alpha \right)=\left[\partial {\omega }_i\left(\alpha \right)/\partial \alpha \right]\delta \alpha$, which implies that:

$$\hspace{1em}\frac{\partial R\left[u\left(x\right);\alpha \right]}{\partial {\alpha }_{j_1}}={\displaystyle \sum _{i=1}^{TF}\hspace{1em}\frac{\partial R\left[u\left(x\right);\alpha \right]}{\partial f_i}}\frac{\partial f_i\left(\alpha \right)}{\partial {\alpha }_{j_1}}.$$

(57)

The features of the 1st-FASAM will be illustrated in Section 5, below, by means of a paradigm neutron diffusion model which admits closed-form exact expressions for all quantities involved (i.e., forward dependent variable, model response, and all sensitivities). The considerable reduction in the number of large-scale computations when using the “function/feature adjoint sensitivity analysis methodology” will become apparent for the computation of second-order (and, subsequently, higher-order) sensitivities, as will be shown in Section 5 and Section 6, below.

4. Illustrative Application of the 1st-FASAM to a Paradigm Particle Transport Model

The application of the 1st-FASAM will be illustrated in this section by considering the sensitivity analysis of an illustrative computational model of a dosimetric response (e.g., energy deposition) stemming from a shielded volumetric source of directly and/or indirectly ionizing radiation. Such models typically describe the radiation and/or particle transport within shielded containers/drums holding vitrified radioactive waste material for deep underground disposal. The distribution of gamma-ray photons emitted by waste products within a shielded vitrified container is computed using sophisticated time-dependent three-dimensional energy-dependent radiation decay and transport models. To illustrate the application of the 1st-FASAM, however, the radiation transport model will be greatly simplified by considering that all processes are steady-state, the radiation is monoenergetic, the sources of radiation are additive, and the geometry is one-dimensional (slab) rather than three-dimensional. These drastic simplifications will enable the analytical solving of the transport equation, thereby minimizing cumbersome algebraic manipulations while highlighting the application of the fundamental concepts underlying the 1st-FASAM.

Consider that a total of “MQ” radioactive sources, each emitting $q_i,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MQ$ particles per unit time (e.g., “second”) per unit area (e.g., “cm²”) are homogeneously distributed (vitrified) within a slab of homogeneous material, which will be designated as “Material A”. There are many radioactive sources (i.e., MQ is a large number); even after several years of “cooling” there remain long-lived radionuclides which require adequate shielding. Included among the most relevant for shielding of waste disposal containers are 11 fission products, 12 actinides, and 5 activation products [10]. The vitrified containment material, designated as Material A, is considered to be homogeneous, comprising “MA” nuclides/elements. Each of these nuclides is characterized by a total microscopic cross section denoted as ${\sigma }_t^{\left(A,i\right)}$, $i=1,\dots ,MA$, and an isotopic number density denoted as $N^{\left(A,i\right)}$, $i=1,\dots ,MA$, where $MA$ denotes the “total number of elements/nuclides within the glassy “Material A”. Geometrically, consider that Material A is a slab of a thickness 2a (cm), which is imprecisely known due to manufacturing tolerances. The slab of Material A, which contains radioactive sources, is considered to be shielded on each of its sides by another slab, made of “Material B”, of thickness b (cm), which is also imprecisely known due to manufacturing tolerances. “Material B” is considered to comprise a total number of $MB$ nuclides/elements, each of which is characterized by a total microscopic cross section denoted as ${\sigma }_t^{\left(B,i\right)}$, $i=1,\dots ,MB$, and an isotopic number density denoted as $N^{\left(B,i\right)}$, $i=1,\dots ,MB$. For example, “Material B” could be magnetite concrete (approximate density: 3.53 g/cm³), composed of $MB=12$ nuclides/elements [11], with approximate partial densities in (g/cm³) provided in parentheses, as follows: 1. Hydrogen (0.011); 2. Oxygen (1.168); 3. Silicon (0.091); 4. Calcium (0.251); 5. Magnesium (0.033); 6. Aluminum (0.083); 7. Sulfur (0.005); 8. Iron (1.676); 9. Titanium (0.192); 10. Chromium (0.006); 11. Manganese (0.007); and 12. Vanadium (0.011).

The distribution of monoenergetic particles within the aforementioned arrangement of slabs is symmetrical with respect to the midplane of the inner slab, so the center of the coordinate system can be chosen at this midplane. Within the inner slab, the uncollided angular flux of monoenergetic neutral particles (e.g., photons and/or neutrons), denoted as ${\phi }_A\left(z,\omega \right)$, in the positive direction perpendicular to the slab, labeled as “direction $z>0$”, is modeled by the following form of the particle transport equation:

$$\omega \frac{d{\phi }_A\left(z,\omega \right)}{dz}+{\mu }_A{\phi }_A\left(z,\omega \right)=Q,0<z<a,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(58)

In Equation (58), the source term is defined as follows:

$$Q\triangleq {\displaystyle \sum _{i=1}^{MQ}{q}_i}.$$

(59)

Also, in Equation (58), the quantity $\omega$ denotes the cosine of the angle between the particle’s direction and the $z$-axis, while ${\mu }_A$ denotes the interaction coefficient of the particles with the inner slab’s homogenized material and is defined as follows:

$${\mu }_A\triangleq {\displaystyle \sum _{i=1}^{MA}{N}^{\left(A,i\right)}{\sigma }_t^{\left(A,i\right)}}.$$

(60)

Due to the model’s symmetry, choosing the origin of the coordinate system at the inner slab’s midplane, the appropriate boundary condition for the inner slab is:

$$\frac{d{\phi }_A\left(z,\omega \right)}{dz}=0,\mathrm{at} \hspace{0.17em}\hspace{0.17em}z=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(61)

The solution of Equations (58) and (61) is the following constant function:

$${\phi }_A\left(z,\omega \right)=\frac{Q}{{\mu }_A},0<z<a,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(62)

The particle transport equation governing the uncollided angular flux of monoenergetic particles, denoted as $\phi \left(z,\omega \right)$, through the outer slab (“Material B”) in particle directions $0\le \omega \le 1$ has the following form:

$$\omega \frac{d\phi \left(z,\omega \right)}{dz}+{\mu }_B\phi \left(z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}a<z<b,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(63)

where the interaction coefficient ${\mu }_B$ is defined as follows:

$${\mu }_B\triangleq {\displaystyle \sum _{i=1}^{MB}{N}^{\left(B,i\right)}{\sigma }_t^{\left(B,i\right)}}.$$

(64)

The uncollided flux is continuous across the interface at $z=a$, i.e.,

$$\phi \left(a,\omega \right)={\phi }_A\left(a,\omega \right)=\frac{Q}{{\mu }_A}.$$

(65)

For subsequent verification of the expressions to be obtained for the response sensitivities to parameters, the closed-form expression of the uncollided flux $\phi \left(z,\omega \right)$, which is the solution of (63) and (65), is provided below:

$$\phi \left(z,\omega \right)=\frac{Q}{{\mu }_A}\exp \left[\frac{{\mu }_B}{\omega }\left(a-z\right)\right],\hspace{0.17em}a\le z\le b,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(66)

The “model response” of interest for this illustrative paradigm model is the total flux of particles interacting with a “kerma response” detector [11] placed on the outer slab’s external surface, which can be represented mathematically by the following expression:

$$R\left[\phi \left(z,\omega \right);\alpha \right]\triangleq {\mu }_D{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)\delta \left(z-b\right)dz}\hspace{0.17em}.$$

(67)

The interaction coefficient, ${\mu }_D$, which characterizes the detector’s material, is defined as follows:

$${\mu }_D\triangleq {\displaystyle \sum _{i=1}^{MD}{N}^{\left(D,i\right)}{\sigma }_t^{\left(D,i\right)}},$$

(68)

where the quantity $MD$ denotes the number of distinct nuclides/elements comprising the detector’s response function, ${\sigma }_t^{\left(D,i\right)}$ denotes the total microscopic interaction cross section of the respective nuclide with the source particles, and $N^{\left(D,i\right)}$ denotes the respective nuclide’s atomic number density, for $i=1,\dots ,MD$.

Substituting the expression of $\phi \left(z,\omega \right)$ obtained in Equation (66) into Equation (67) yields the following expression for the response $R\left[\phi \left(z,\omega \right);\alpha \right]$:

$$R\left(\phi ;\alpha \right)=\frac{Q{\mu }_D}{{\mu }_A}{E}_2\left[\left(b-a\right){\mu }_B\right],$$

(69)

where the exponential-integral function $E_n\left(z\right)$ is defined as follows:

$$E_n\left(z\right)\triangleq {\displaystyle {\int }_0^1{y}^{n-2}{e}^{-z/y}dy};E_0\left(z\right)=\frac{e^{-z}}{z};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{E}_1\left(z\right)\triangleq {\displaystyle {\int }_0^1{y}^{-1}{e}^{-z/y}dy=}-Ei\left(-z\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{E}_2\left(0\right)=1;$$

(70)

and possesses the following important properties:

$$E_n\left(z\right)=\frac{1}{n-1}\left[e^{-z}-z{E}_{n-1}\left(z\right)\right],\hspace{0.17em}\hspace{0.17em}for\hspace{0.17em}\hspace{0.17em}n>1;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\frac{d{E}_n\left(z\right)}{dz}=-E_{n-1}\left(z\right);\hspace{0.17em}\hspace{0.17em}{E}_n\left(z\right)={\displaystyle {\int }_z^{\infty }{E}_{n-1}\left(y\right)dy}\hspace{0.17em}.$$

(71)

The uncertain parameters that describe the properties (microscopic cross sections, atomic number densities, source, and slab boundaries) characterizing the two materials and the detector will be generically denoted as ${\alpha }_i,\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TP$, where $TP$ denotes the “total number of model parameters.” These parameters will be considered to be the components of the (column) “vector of imprecisely known model parameters”, defined as follows:

$$\begin{array}{l}\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}\triangleq \left[{\sigma }_t^{\left(A,i\right)},N^{\left(A,i\right)};i=1,\dots ,MA\hspace{0.17em}\hspace{0.17em};{\sigma }_t^{\left(B,i\right)},N^{\left(B,i\right)};i=1,\dots ,MB\right.;\\ {\left.{\sigma }_t^{\left(D,i\right)},N^{\left(D,i\right)};i=1,\dots ,MD;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{q}_1,\dots ,q_{MQ};a,b\right]}^{†};\hspace{0.17em}\hspace{0.17em}TP=2MA+2MB+2MD+MQ+2.\end{array}$$

(72)

From the definitions provided in Equations (59), (60), (64) and (68), it follows that the interaction coefficients and the source are functions of the vector $\alpha$ of model parameters.

4.1. Current State-of-the Art Procedure: Application of the 1st-CASAM to Compute First-Order Response Sensitivities Directly with Respect to Model Parameters

The total first-order sensitivity of the response defined in Equation (67) to the model’s primary parameters (including interface and boundary locations) is provided by the Gateaux- (G-) differential ${\left\{\delta R\left(\phi ;\alpha ;\delta \phi ;\delta \alpha \right)\right\}}_{{\alpha }^0}$ of the response $R\left(\phi ;\alpha \right)$ for arbitrary variations $\left(\delta \phi ,\delta \alpha \right)$ around the nominal parameter and state functions values, which is defined as follows:

$$\begin{array}{l}\delta R{\left(\phi ;\alpha ;\delta \phi ;\delta \alpha \right)}_{{\alpha }^0}\triangleq \frac{d}{d\epsilon }\left\{\left[{\displaystyle \sum _{i=1}^{MD}\left(N^{\left(D,i\right)}+\epsilon \delta N^{\left(D,i\right)}\right)}\left({\sigma }_t^{\left(D,i\right)}+\epsilon \delta {\sigma }_t^{\left(D,i\right)}\right)\right.\right.\\ \times {\left.{\left.{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a+\epsilon \delta a}{\overset{b+\epsilon \delta b}{\int }}\left(\phi +\epsilon \delta \phi \right)\delta \left(z-b-\epsilon \delta b\right)}}\hspace{0.17em}\right]}_{{\alpha }^0}dz\right\}}_{\epsilon =0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left\{\delta R{\left(\phi ;\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R{\left(\phi ;\alpha ;\delta \phi \right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(73)

where the “direct-effect” term ${\left\{\delta R{\left(\phi ;\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}$ depends only on variations $\delta \alpha$ in the parameters, and is defined as follows:

$$\begin{array}{l}{\left\{\delta R{\left(\phi ;\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}\triangleq \left\{{\displaystyle \sum _{i=1}^{MD}\left(N^{\left(D,i\right)}\delta {\sigma }_t^{\left(D,i\right)}+{\sigma }_t^{\left(D,i\right)}\delta N^{\left(D,i\right)}\right)}\right.\\ \times {\left.{\displaystyle \underset{0}{\overset{1}{\int }}d\omega \hspace{0.17em}{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}\phi \left(z,\omega \right)\delta \left(z-b\right)}}\right\}}_{{\alpha }^0}-\left(\delta b\right){\left\{{\mu }_D{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)}\hspace{0.17em}}{\delta }^{\prime }\left(z-b\right)dz\right\}}_{{\alpha }^0},\end{array}$$

(74)

and where the “indirect-effect” term ${\left\{\delta R{\left(\phi ;\alpha ;\delta \phi \right)}_{{\alpha }^0}\right\}}_{ind}$ depends only on variations $\delta \phi \left(z,\omega \right)$ in the dependent variable (state function) and is defined as follows:

$${\left\{\delta R{\left(\phi ;\alpha ;\delta \phi \right)}_{{\alpha }^0}\right\}}_{ind}\triangleq {\left\{{\mu }_D\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\delta \phi \left(z,\omega \right)\hspace{0.17em}}}\delta \left(z-b\right)dz\right\}}_{{\alpha }^0}.$$

(75)

Since the flux $\phi \left(z,\omega \right)$ is known, the direct-effect term in Equation (74) can be computed immediately at this stage, for any parameter variations that appear in its expression. On the contrary, the “indirect-effect” terms in Equation (75) cannot be computed at this stage since the 1st-order variation, $\delta \phi \left(z,\omega \right)$, is not available at this stage. The variational function $\delta \phi \left(z,\omega \right)$ could be determined by solving the 1st-Level Variational Sensitivity System (1st-LVSS), which is obtained by G-differentiating Equations (63) and (65) at the nominal parameter values, to obtain the following system of equations:

$${\left\{\frac{d}{d\epsilon }{\left[\omega \frac{d\left(\phi +\epsilon \delta \phi \right)}{dz}+{\mu }_B\left(\alpha +\epsilon \delta \alpha \right)\left(\phi +\epsilon \delta \phi \right)\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}=0,\hspace{0.17em}{a}^0<z<b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1;$$

(76)

$${\left\{\frac{d}{d\epsilon }\left[\phi \left(z=a^0+\epsilon \delta a;\omega \right)+\epsilon \delta \phi \left(z=a^0+\epsilon \delta a;\omega \right)\right]\right\}}_{\epsilon =0}={\left\{\frac{d}{d\epsilon }{\left[\frac{Q\left(\alpha +\epsilon \delta \alpha \right)}{{\mu }_A\left(\alpha +\epsilon \delta \alpha \right)}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}.$$

(77)

Carrying out the operations with respect to $\epsilon$ in (76) and (77) yields the following 1st-LVSS to be satisfied by the 1st-order forward variational function $\delta \phi \left(z,\omega \right)$:

$$\begin{array}{l}{\left\{\omega \frac{d}{dz}\left[\delta \phi \left(z,\omega \right)\right]+{\mu }_B\left(\alpha \right)\left[\delta \phi \left(z,\omega \right)\right]\right\}}_{{\alpha }^0}=-{\left\{\phi \left(z,\omega \right){\displaystyle \sum _{i=1}^{MB}\frac{\partial {\mu }_B\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}\right\}}_{{\alpha }^0};\\ a^0<z<b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1;\end{array}$$

(78)

$$\begin{gathered} {\left\{\delta \phi \left(z,\omega \right)\right\}}_{z=a^0}={\left\{\frac{1}{{\mu }_A\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MQ}\frac{\partial Q\left(\alpha \right)}{\partial {\alpha }_i}}\left(\delta {\alpha }_i\right)-\frac{Q\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MA}\frac{\partial {\mu }_A\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}\right\}}_{{\alpha }^0} \\ -\left(\delta a\right){\left\{\frac{d\phi \left(z,\omega \right)}{dz}\right\}}_{z=a^0}={\left\{\frac{1}{{\mu }_A\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MQ}\frac{\partial Q\left(\alpha \right)}{\partial {\alpha }_i}}\left(\delta {\alpha }_i\right)-\frac{Q\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MA}\frac{\partial {\mu }_A\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}\right\}}_{{\alpha }^0} \\ +\left(\delta a\right){\left\{\frac{Q\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}\frac{{\mu }_B\left(\alpha \right)}{\omega }\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}z=a^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1. \end{gathered}$$

(79)

It is evident that solving Equations (78) and (79) would require at least $TP$ large-scale computations (i.e., at least one large-scale computation for each parameter variation) to obtain the 1st-order variational function $v^{\left(1\right)}\left(i;z,\omega \right)$ for every parameter variation. Performing so many large-scale computations would be impractical for large-scale systems involving many parameters.

The alternative to solving repeatedly the 1st-LVSS for every possible variation in the system’s imprecisely known parameters is to use the First-Level Adjoint Sensitivity System (1st-LASS), which is constructed by applying the following sequence of steps:

• Consider that the function $v^{\left(1\right)}\left(i;z,\omega \right)$ belongs to a Hilbert space, which will be denoted as ${\mathsf{H}}_1\left({\mathsf{\Omega }}_z\right),\hspace{0.17em}\hspace{0.17em}{\mathsf{\Omega }}_z\triangleq \left\{\omega \in \left[0,1\right]\otimes z\in \left[a,b\hspace{0.17em}\right]\right\}$. This Hilbert space is endowed with an inner product of two functions $\phi \left(z,\omega \right)\in {\mathsf{H}}_1$ and $\psi \left(z,\omega \right)\in {\mathsf{H}}_1$, which is denoted as ${〈\phi \left(z,\omega \right),\psi \left(z,\omega \right)〉}_1$ and is defined as follows:

  $${〈\phi \left(z,\omega \right),\psi \left(z,\omega \right)〉}_1\triangleq {\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)\psi \left(z,\omega \right)}}\hspace{0.17em}dz\right\}}_{{\alpha }^0}.$$

  (80)
• Using the definition provided in Equation (80), construct the inner product of a function $a^{\left(1\right)}\left(z,\omega \right)\in {\mathsf{H}}_1$ with Equation (78) to obtain:

  $$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\left[\omega \frac{d}{dz}\delta \phi \left(z,\omega \right)+{\mu }_B\left(\alpha \right)\delta \phi \left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=-{\left\{{\displaystyle \sum _{i=1}^{MB}\frac{\partial {\mu }_B\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}.\end{array}$$

  (81)
• Integrate by parts, over the independent variable z, the left side of Equation (81) to obtain

  $$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\left[\omega \frac{d}{dz}\delta \phi \left(z,\omega \right)+{\mu }_B\left(\alpha \right)\delta \phi \left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }{\left\{\hspace{0.17em}{a}^{\left(1\right)}\left(b,\omega \right)\delta \phi \left(b,\omega \right)-\hspace{0.17em}{a}^{\left(1\right)}\left(a,\omega \right)\delta \phi \left(a,\omega \right)\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\delta \phi \left(z,\omega \right)\left[-\omega \frac{d}{dz}{a}^{\left(1\right)}\left(z,\omega \right)+{\mu }_B\left(\alpha \right)a^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}.\end{array}$$

  (82)
• Require the last term on the right side of Equation (82) to represent the indirect-effect term defined in Equation (75) by imposing the following relationship:

  $$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\delta \phi \left(z,\omega \right)\left[-\omega \frac{d}{dz}{a}^{\left(1\right)}\left(z,\omega \right)+{\mu }_B\left(\alpha \right)\hspace{0.17em}{a}^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\hspace{0.17em}{\left\{{\mu }_D\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\delta \phi \left(z,\omega \right)\hspace{0.17em}}}\delta \left(z-b\right)dz\right\}}_{{\alpha }^0}.\end{array}$$

  (83)
• The relation in Equation (83) implies that the following equation holds, in the weak sense, at the nominal parameter values:

  $${\left\{-\omega \frac{d}{dz}{a}^{\left(1\right)}\left(z,\omega \right)+{\mu }_B\left(\alpha \right)a^{\left(1\right)}\left(z,\omega \right)\right\}}_{{\alpha }^0}={\left\{{\mu }_D\left(\alpha \right)\delta \left(z-b\right)\right\}}_{{\alpha }^0}\hspace{0.17em},\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

  (84)
• Complete the definition of the function $a^{\left(1\right)}\left(z,\omega \right)$ by requiring that the unknown value of the function $\delta \phi \left(b,\omega \right)$ be eliminated from appearing on the right side of Equation (82). This requirement is met by imposing the following condition to be satisfied by the function $a^{\left(1\right)}\left(z,\omega \right)$ on the model’s outer boundary:

  $$a^{\left(1\right)}\left(z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

  (85)

Altogether, Equations (84) and (85) constitute a well-posed system, called the “First-Level Adjoint Sensitivity System” (1st-LASS), for determining the function $\hspace{0.17em}{a}^{\left(1\right)}\left(z,\omega \right)\in {\mathsf{H}}_1$. The function $\hspace{0.17em}{a}^{\left(1\right)}\left(z,\omega \right)\in {\mathsf{H}}_1$ is called the “first-level adjoint sensitivity function.” Since the 1st-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain the 1st-level adjoint sensitivity function $\hspace{0.17em}{\psi }_1\left(z,\omega \right).$ For the paradigm model under consideration, the 1st-LASS given by Equations (84) and (85) can be solved exactly to obtain the following closed-form expression for the 1st-level adjoint function $a^{\left(1\right)}\left(z,\omega \right)$:

$$a^{\left(1\right)}\left(z,\omega \right)={\left\{\frac{{\mu }_D\left(\alpha \right)}{\omega }\left[1-H\left(z-b\right)\right]\exp \left[\frac{{\mu }_B\left(\alpha \right)\left(z-b\right)}{\omega }\right]\right\}}_{{\alpha }^0},\hspace{0.17em}$$

(86)

where $H\left(z-b\right)$ denotes the Heaviside functional, defined as follows:

$$H\left(z-b\right)=\left\{\begin{array}{c}1,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z\ge b;\\ 0,\hspace{0.17em}\hspace{0.17em}z<b.\end{array}\right.$$

(87)

Collecting the results obtained in Equation (81) through (85) leads to the following expression for the indirect-effect term:

$$\begin{array}{l}{\left\{\delta R{\left(\phi ;\alpha ;\delta \phi \right)}_{{\alpha }^0}\right\}}_{ind}=\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega \hspace{0.17em}\hspace{0.17em}{a}^{\left(1\right)}\left(a,\omega \right)}\right.\left[\frac{1}{{\mu }_A\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MQ}\frac{\partial Q\left(\alpha \right)}{\partial {\alpha }_i}}\left(\delta {\alpha }_i\right)\right.\\ {\left.\left.-\frac{Q\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MA}\frac{\partial {\mu }_A\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}+\hspace{0.17em}\left(\delta a\right)\frac{Q\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}\frac{{\mu }_B\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}\\ -{\left\{{\displaystyle \sum _{i=1}^{MB}\frac{\partial {\mu }_B\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{b_1}{\overset{b_2}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\triangleq {\left\{\delta R\left(\phi ;a^{\left(1\right)};\alpha ;\hspace{0.17em}\delta \alpha \right)\right\}}_{ind}.\end{array}$$

(88)

The appearance of the 1st-level adjoint function $a^{\left(1\right)}\left(z,\omega \right)$ in the list of arguments of the indirect-effect term ${\left\{\delta R\left(\phi ;a^{\left(1\right)};\alpha ;\hspace{0.17em}\delta \alpha \right)\right\}}_{ind}$ in Equation (88) emphasizes the fact that the function $\delta \phi \left(z,\omega \right)$, which depends implicitly on parameter variations, has been eliminated (and has been replaced by the dependence on the first-level adjoint sensitivity function $a^{\left(1\right)}$). The indirect-effect term is now expressed in terms of the 1st-level adjoint function $a^{\left(1\right)}\left(z,\omega \right)$, which does not depend on the model parameter variations. Hence, the 1st-LASS needs to be solved just once (which means a single “large-scale computation” comparable to solving the original equations underlying the model) to obtain the 1st-level adjoint function $a^{\left(1\right)}\left(z,\omega \right)$. After obtaining $a^{\left(1\right)}\left(z,\omega \right)$, all of the partial sensitivities included in the indirect-effect term, cf. Equation (88), are obtained by quadratures for evaluating the integrals involving $a^{\left(1\right)}\left(z,\omega \right)$.

Adding the results obtained in Equations (88) and (74) yields the following complete expression for the total 1st-order response sensitivity ${\left\{\delta R\left(\phi ;\alpha ;a^{\left(1\right)};\delta \alpha \right)\right\}}_{{\alpha }^0}$:

$$\begin{array}{l}{\left\{\delta R\left(\phi ;\alpha ;a^{\left(1\right)};\delta \alpha \right)\right\}}_{{\alpha }^0}=-\left(\delta b\right){\left\{{\mu }_D{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)}\hspace{0.17em}}{\delta }^{\prime }\left(z-b\right)dz\right\}}_{{\alpha }^0}\\ +\left\{{\displaystyle \sum _{i=1}^{M_D}\left(N^{\left(D,i\right)}\delta {\sigma }_t^{\left(D,i\right)}+{\sigma }_t^{\left(D,i\right)}\delta N^{\left(D,i\right)}\right)}\right.{\left.{\displaystyle \underset{0}{\overset{1}{\int }}d\omega \hspace{0.17em}{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}\phi \left(z,\omega \right)\delta \left(z-b\right)}}\right\}}_{{\alpha }^0}\\ +\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega \hspace{0.17em}\hspace{0.17em}{a}^{\left(1\right)}\left(a,\omega \right)}\right.\left[\frac{1}{{\mu }_A\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MQ}\frac{\partial Q\left(\alpha \right)}{\partial {\alpha }_i}}\left(\delta {\alpha }_i\right)-\frac{Q\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{\displaystyle \sum _{i=1}^{MA}\frac{\partial {\mu }_A\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}\right.\\ {\left.\left.+\hspace{0.17em}\left(\delta a\right)\frac{Q\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}\frac{{\mu }_B\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}-{\left\{{\displaystyle \sum _{i=1}^{MB}\frac{\partial {\mu }_B\left(\alpha \right)}{\partial {\alpha }_i}\left(\delta {\alpha }_i\right)}{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{b_1}{\overset{b_2}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}.\end{array}$$

(89)

The specific expression of each 1st-order partial sensitivity of the response $R\left(\phi ;\alpha \right)$ to each uncertain parameter is obtained by identifying the expression that multiplies the respective parameter variation, which yields the following results:

(i)

Sensitivities stemming solely from the direct-effect term:

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial b}\right\}}_{{\alpha }^0}=-{\left\{{\mu }_D\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)}\hspace{0.17em}}{\delta }^{\prime }\left(z-b\right)dz\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=-{\left\{\frac{Q\left(\alpha \right){\mu }_B\left(\alpha \right){\mu }_D\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}{E}_1\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\end{array}.$$

(90)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(D,i\right)}}\right\}}_{{\alpha }^0}={\left\{N^{\left(D,i\right)}{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}\phi \left(z,\omega \right)\delta \left(z-b\right)}}\right\}}_{{\alpha }^0}\\ ={\left\{\frac{Q\left(\alpha \right)N^{\left(D,i\right)}}{{\mu }_A}{E}_2\left[\left(b-a\right){\mu }_B\right]\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MD;\end{array}$$

(91)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(D,i\right)}}\right\}}_{{\alpha }^0}={\left\{{\sigma }_t^{\left(D,i\right)}{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}\phi \left(z,\omega \right)\delta \left(z-b\right)}}\right\}}_{{\alpha }^0}\\ ={\left\{\frac{Q\left(\alpha \right){\sigma }_t^{\left(D,i\right)}}{{\mu }_A}{E}_2\left[\left(b-a\right){\mu }_B\right]\right\}}_{{\alpha }^0}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MD;\end{array};$$

(92)

(ii)

Sensitivities stemming solely from the indirect-effect term:

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(A,i\right)}}\right\}}_{{\alpha }^0}=-{\left\{{\sigma }_t^{\left(A,i\right)}\frac{Q\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\hspace{0.17em}\omega \hspace{0.17em}d\omega \right\}}_{{\alpha }^0}\\ =-{\left\{{\sigma }_t^{\left(A,i\right)}\frac{Q\left(\alpha \right){\mu }_D\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{E}_2\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MA;\end{array}$$

(93)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(A,i\right)}}\right\}}_{{\alpha }^0}=-{\left\{N^{\left(A,i\right)}\frac{Q\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\hspace{0.17em}\omega \hspace{0.17em}d\omega \right\}}_{{\alpha }^0}\\ =-{\left\{N^{\left(A,i\right)}\frac{Q\left(\alpha \right){\mu }_D\left(\alpha \right)}{{\mu }_A^2\left(\alpha \right)}{E}_2\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MA;\end{array}$$

(94)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(B,i\right)}}\right\}}_{{\alpha }^0}=-{\left\{{\sigma }_t^{\left(B,i\right)}{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\\ ={\left\{{\sigma }_t^{\left(B,i\right)}\frac{Q\left(\alpha \right){\mu }_D\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}\left(a-b\right)E_1\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MB;\end{array}$$

(95)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(B,i\right)}}\right\}}_{{\alpha }^0}=-{\left\{N^{\left(B,i\right)}{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\\ ={\left\{N^{\left(B,i\right)}\frac{Q\left(\alpha \right){\mu }_D\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}\left(a-b\right)E_1\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MB;\end{array}$$

(96)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial q_i}\right\}}_{{\alpha }^0}={\left\{\frac{1}{{\mu }_A\left(\alpha \right)}\hspace{0.17em}{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\hspace{0.17em}\omega \hspace{0.17em}d\omega \right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left\{\frac{{\mu }_D\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}{E}_2\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MQ;\end{array}$$

(97)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial a}\right\}}_{{\alpha }^0}={\left\{\frac{Q\left(\alpha \right){\mu }_B\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\hspace{0.17em}d\omega \right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left\{\frac{Q\left(\alpha \right){\mu }_B\left(\alpha \right){\mu }_D\left(\alpha \right)}{{\mu }_A\left(\alpha \right)}{E}_1\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(98)

It is noteworthy that the absolute sensitivities $\partial R\left(\phi ;\alpha \right)/\partial q_i$ all have equal values, i.e.,$\partial R\left(\phi ;\alpha \right)/\partial q_1=\dots =\partial R\left(\phi ;\alpha \right)/\partial q_{MQ}$. Also noteworthy are the results indicating that the relative sensitivities of the response with respect to the nuclide densities have the same values as the relative sensitivities of the response with respect to the microscopic cross sections, i.e.,

$${\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(A,i\right)}}\frac{N^{\left(A,i\right)}}{R\left(\phi ;\alpha \right)}\right\}}_{{\alpha }^0}={\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(A,i\right)}}\frac{{\sigma }_t^{\left(A,i\right)}}{R\left(\phi ;\alpha \right)}\right\}}_{{\alpha }^0}=-{\left\{\frac{{\sigma }_t^{\left(A,i\right)}{N}^{\left(A,i\right)}}{{\mu }_A}\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MA;$$

(99)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(B,i\right)}}\frac{N^{\left(B,i\right)}}{R\left(\phi ;\alpha \right)}\right\}}_{{\alpha }^0}={\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(B,i\right)}}\frac{{\sigma }_t^{\left(B,i\right)}}{R\left(\phi ;\alpha \right)}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left\{{\sigma }_t^{\left(B,i\right)}{N}^{\left(B,i\right)}\left(a-b\right)\frac{E_1\left[\left(b-a\right){\mu }_B\left(\alpha \right)\right]}{E_2\left[\left(b-a\right){\mu }_B\right]}\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MB;\end{array}$$

(100)

$${\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(D,i\right)}}\frac{N^{\left(D,i\right)}}{R\left(\phi ;\alpha \right)}\right\}}_{{\alpha }^0}=\hspace{0.17em}\hspace{0.17em}{\left\{\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(D,i\right)}}\frac{{\sigma }_t^{\left(D,i\right)}}{R\left(\phi ;\alpha \right)}\right\}}_{{\alpha }^0}=\hspace{0.17em}{\left\{\frac{N^{\left(D,i\right)}{\sigma }_t^{\left(D,i\right)}}{{\mu }_D\left(\alpha \right)}\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,N_D.$$

(101)

Notably, obtaining all of the 1st-order partial response sensitivities to the primary model parameters has necessitated a single “large-scale” computation for determining the 1st-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. This is in contradistinction with the use of the 1st-LVSS, which would require at least $TP$ large-scale computations to obtain the corresponding $TP$ first-order sensitivities. Furthermore, the expressions obtained for these sensitivities are exact, as opposed to being approximate, as would have been the case if these sensitivities had been computed by finite-difference or statistical procedures. Also, many more large-scale computations would have been necessary to compute these 1st-order partial response sensitivities by any other (finite-difference or statistical) procedure. Thus, the 1st-CASAM is the most efficient computational method for obtaining the exact expressions of the 1st-order partial response sensitivities to the primary model parameters.

The fact that so many relative sensitivities have the same values, as indicated in Equations (99)–(101), respectively, would render quasi-useless the statistical methods that attempt to rank the sensitivities by their relative values, because such statistical methods break down in situations when many relative sensitivities have equal values. On the other hand, if statistical methods attempted to rank sensitivities by their absolute values, these statistical methods would fail when attempting to determine the response sensitivities to the material properties of the detector. In all cases, the only reliable method for computing response sensitivities accurately (and also most efficiently, from a computational standpoint) is the 1st-CASAM.

4.2. Most Efficient Alternative Procedure: Applying the 1st-FASAM to Compute First-Order Response Sensitivities to Functions/Features of Model Parameters

The particle transport equation governing the uncollided angular flux $\phi \left(z,\omega \right)$, i.e., Equations (63) and (65), can be written in the following form, which highlights the “feature” functions of the primary model parameters:

$$\omega \frac{d\phi \left(z,\omega \right)}{dz}+g_2\left(\alpha \right)\phi \left(z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}a<z<b,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1;$$

(102)

$$\phi \left(a,\omega \right)=\hspace{0.17em}{g}_1\left(\alpha \right);$$

(103)

$$R\left[\phi \left(z,\omega \right);f\left(\alpha \right)\right]\triangleq h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)\delta \left(z-b\right)dz}\hspace{0.17em}.$$

(104)

where:

$$f\left(\alpha \right)\triangleq {\left[g\left(\alpha \right);h\left(\alpha \right);\lambda \left(\alpha \right);\omega \left(\alpha \right)\right]}^{†}\triangleq {\left[f_1\left(\alpha \right),\dots ,f_{TF}\left(\alpha \right)\right]}^{†};\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}TF\triangleq 2+1+2;$$

(105)

$$\begin{array}{l}g\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right)\right]}^{†};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{g}_1\left(\alpha \right)\triangleq Q\left(\alpha \right){\left[{\displaystyle \sum _{i=1}^{MA}{N}^{\left(A,i\right)}{\sigma }_t^{\left(A,i\right)}}\right]}^{-1};\hspace{0.17em}\\ g_2\left(\alpha \right)\triangleq {\mu }_B\left(\alpha \right)={\displaystyle \sum _{i=1}^{MB}{N}^{\left(B,i\right)}{\sigma }_t^{\left(B,i\right)}};\hspace{0.17em}h\left(\alpha \right)\triangleq {\mu }_D\left(\alpha \right)={\displaystyle \sum _{i=1}^{MD}{N}^{\left(D,i\right)}{\sigma }_t^{\left(D,i\right)}};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\lambda \left(\alpha \right)\triangleq a\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\omega \left(\alpha \right)\triangleq b\hspace{0.17em};\end{array}$$

(106)

In terms of the components of the “feature” function $f\left(\alpha \right)$, the uncollided flux and the model response have the following expressions:

$$\phi \left(z,\omega \right)=g_1\left(\alpha \right)\exp \left[\frac{\hspace{0.17em}{g}_2\left(\alpha \right)}{\omega }\left(a-z\right)\right],\hspace{0.17em}a\le z\le b,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(107)

$$R\left(\phi ;\alpha \right)=g_1\left(\alpha \right)h\left(\alpha \right)E_2\left[\left(b-a\right)\hspace{0.17em}{g}_2\left(\alpha \right)\right].$$

(108)

The 1st-order total sensitivity of $R\left(\phi ;f\right)$ is obtained by determining the first-order G-differential $\delta R{\left(\phi ;f;\delta \phi ;\delta f\right)}_{{\alpha }^0}$ of $R\left(\phi ;f\right)$ at the nominal parameter values, which is determined by applying the definition of the G-differential to Equation (104). This operation yields the following expression:

$$\begin{array}{l}\delta R{\left(\phi ;f;v^{\left(1\right)};\delta f\right)}_{{\alpha }^0}\triangleq {\left\{\frac{d}{d\epsilon }{\left[\left(h+\epsilon \delta h\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a+\epsilon \delta a}{\overset{b+\epsilon \delta b}{\int }}\left(\phi +\epsilon v^{\left(1\right)}\right)\delta \left(z-b-\epsilon \delta b\right)dz}}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\hspace{0.17em}\\ \hspace{0.17em}={\left\{\delta R{\left(\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R{\left(\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(109)

where the direct-effect term ${\left\{\delta R{\left(\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}$ depends only on the parameter variations $\delta f$ and is defined as follows:

$$\begin{array}{l}{\left\{\delta R{\left(\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq \left(\delta h\right){\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)\delta \left(z-b\right)dz}}\right\}}_{{\alpha }^0}\\ -\left(\delta b\right){\left\{h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega \phi \left(z,\omega \right)}{\delta }^{\prime }\left(z-b\right)\right\}}_{{\alpha }^0},\end{array}$$

(110)

and where the indirect-effect term ${\left\{\delta R{\left(\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ depends only on the variational function $v^{\left(1\right)}\left(z,\omega \right)$ and is defined as follows:

$${\left\{\delta R{\left(\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\triangleq {\left\{h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{v}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-b\right)dz}\right\}}_{{\alpha }^0}.$$

(111)

The 1st-LVSS to be satisfied by the 1st-order forward variational function $v^{\left(1\right)}\left(z,\omega \right)$ is obtained by G-differentiating Equations (102) and (103), to obtain the following system of equations:

$${\left\{\omega \frac{d{v}^{\left(1\right)}\left(z,\omega \right)}{dz}+g_2\left(\alpha \right)v^{\left(1\right)}\left(z,\omega \right)\right\}}_{{\alpha }^0}=-{\left\{\left(\delta g_2\right)\phi \left(z,\omega \right)\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}^0<z<b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1;$$

(112)

$$\begin{array}{l}{\left\{v^{\left(1\right)}\left(z,\omega \right)\right\}}_{z=a^0}={\left\{\delta g_1\left(\alpha \right)-\left(\delta a\right)g_1\left(\alpha \right){\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=a}\right\}}_{{\alpha }^0}\\ ={\left\{\delta g_1\left(\alpha \right)\right\}}_{{\alpha }^0}+\left(\delta a\right){\left\{\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\right\}}_{{\alpha }^0};\hspace{0.17em}\hspace{0.17em}z=a^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.\end{array}$$

(113)

It is noteworthy that the above 1st-LVSS would need to be solved just three times to obtain the three variational functions $v^{\left(1\right)}\left(i;z,\omega \right),\hspace{0.17em}\hspace{0.17em}i=1,2,3,$ which would correspond to the variations $\delta g_1$, $\delta g_2$, and $\delta a$; thus, $v^{\left(1\right)}\left(1;z,\omega \right)$ would be obtained by setting $\delta g_1=1$, $\delta g_2=0$, and $\delta a=0$; $v^{\left(1\right)}\left(2;z,\omega \right)$ would be obtained by setting $\delta g_1=0$, $\delta g_2=1$, and $\delta a=0$; and $v^{\left(1\right)}\left(3;z,\omega \right)$ would be obtained by setting $\delta g_1=0$, $\delta g_2=0$, and $\delta a=1$.

The need for computing the functions $v^{\left(1\right)}\left(i;z,\omega \right),\hspace{0.17em}\hspace{0.17em}i=1,2,3,$ is circumvented by expressing the indirect-effect term ${\left\{\delta R{\left(\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ defined in Equation (111) in terms of a 1st-level adjoint function, which will be denoted as $a^{\left(1\right)}\left(z,\omega \right)$, and which is the solution of a 1st-Level Adjoint Sensitivity System (1st-LASS) constructed by performing the same sequence of operations as indicated in Equations (80)–(82). The resulting 1st-LASS for the 1st-level adjoint function $a^{\left(1\right)}\left(z,\omega \right)$ is the same as was obtained in Equations (84) and (85), namely:

$${\left\{-\omega \frac{d}{dz}{a}^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)a^{\left(1\right)}\left(z,\omega \right)\right\}}_{{\alpha }^0}={\left\{h\left(\alpha \right)\delta \left(z-b\right)\right\}}_{{\alpha }^0}\hspace{0.17em},\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(114)

$$a^{\left(1\right)}\left(z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(115)

The solution of Equations (114) and (115) has the following expression:

$$a^{\left(1\right)}\left(z,\omega \right)={\left\{\frac{h\left(\alpha \right)}{\omega }\left[1-H\left(z-b\right)\right]\exp \left[\frac{g_2\left(\alpha \right)\left(z-b\right)}{\omega }\right]\right\}}_{{\alpha }^0}.$$

(116)

In terms of the 1st-level adjoint function $a^{\left(1\right)}\left(z,\omega \right)$, the indirect-effect term ${\left\{\delta R{\left(\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ will have the following expression:

$$\begin{array}{l}{\left\{\delta R{\left(\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}={\left\{{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\left[\left(\delta g_1\right)+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\hspace{0.17em}\right]\hspace{0.17em}\omega d\omega \right\}}_{{\alpha }^0}\\ -{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}.\end{array}$$

(117)

Adding the expression for the indirect-effect term obtained in Equation (117) with the expression of the direct-effect term obtained in Equation (110) yields the following expression for the total 1st-order G-differential $\delta R{\left(\phi ;f;\delta f\right)}_{{\alpha }^0}$:

$$\begin{array}{l}\delta R{\left(\phi ;f;\delta f\right)}_{{\alpha }^0}=\left(\delta h\right){\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)\delta \left(z-b\right)dz}}\right\}}_{{\alpha }^0}-\left(\delta b\right){\left\{h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega \phi \left(z,\omega \right)}{\delta }^{\prime }\left(z-b\right)\right\}}_{{\alpha }^0}\\ -{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\left[\left(\delta g_1\right)+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\hspace{0.17em}\right]\hspace{0.17em}\omega d\omega \right\}}_{{\alpha }^0}.\end{array}$$

(118)

It follows from Equation (118) that the expressions of the partial sensitivities of the response $R\left(\phi ;\alpha \right)$ with respect to the components of the “feature” function $f\left(\alpha \right)$ are as follows:

$${\left\{\frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}\right\}}_{{\alpha }^0}={\left\{{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\hspace{0.17em}\omega \hspace{0.17em}d\omega \right\}}_{{\alpha }^0}={\left\{h\left(\alpha \right)E_2\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0};$$

(119)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)}\right\}}_{{\alpha }^0}=-{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left\{g_1\left(\alpha \right)h\left(\alpha \right)\left(a-b\right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\end{array}$$

(120)

$${\left\{\frac{\partial R\left(\phi ;f\right)}{\partial h\left(\alpha \right)}\right\}}_{{\alpha }^0}={\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)\delta \left(z-b\right)dz}}\right\}}_{{\alpha }^0}={\left\{g_1\left(\alpha \right)E_2\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0};$$

(121)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;f\right)}{\partial a}\right\}}_{{\alpha }^0}={\left\{g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(a,\omega \right)}\hspace{0.17em}d\omega \right\}}_{{\alpha }^0}\\ \hspace{1em}\hspace{1em}={\left\{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0};\end{array}$$

(122)

$$\begin{array}{l}{\left\{\frac{\partial R\left(\phi ;f\right)}{\partial b}\right\}}_{{\alpha }^0}=-{\left\{h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)}\hspace{0.17em}}{\delta }^{\prime }\left(z-b\right)dz\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=-{\left\{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(123)

The above expressions can be used to obtain efficiently the sensitivities of the response with respect to the primary model parameters by using the chain-rule of differentiation, as follows:

$$\frac{\partial R\left(\phi ;\alpha \right)}{\partial q_i}=\frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}\frac{\partial g_1\left(\alpha \right)}{\partial q_i}=\frac{1}{{\mu }_A\left(\alpha \right)}\frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,MQ;$$

(124)

$$\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(A,i\right)}}=\frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}\frac{\partial g_1\left(\alpha \right)}{\partial N^{\left(A,i\right)}}=-\frac{q{\sigma }_t^{\left(A,i\right)}}{{\mu }_A^2\left(\alpha \right)}\frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,M_A;$$

(125)

$$\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(A,i\right)}}=\frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}\frac{\partial g_1\left(\alpha \right)}{\partial {\sigma }_t^{\left(A,i\right)}}=-\frac{q{N}^{\left(A,i\right)}}{{\mu }_A^2\left(\alpha \right)}\frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,M_A;$$

(126)

$$\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(B,i\right)}}=\frac{\partial R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)}\frac{\partial g_2\left(\alpha \right)}{\partial N^{\left(B,i\right)}}={\sigma }_t^{\left(B,i\right)}\frac{\partial R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,M_B;$$

(127)

$$\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(B,i\right)}}=\frac{\partial R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)}\frac{\partial g_2\left(\alpha \right)}{\partial {\sigma }_t^{\left(B,i\right)}}=N^{\left(B,i\right)}\hspace{0.17em}\frac{\partial R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,M_B;$$

(128)

$$\frac{\partial R\left(\phi ;\alpha \right)}{\partial N^{\left(D,i\right)}}=\frac{\partial R\left(\phi ;f\right)}{\partial h\left(\alpha \right)}\frac{\partial h\left(\alpha \right)}{\partial N^{\left(D,i\right)}}={\sigma }_t^{\left(D,i\right)}\frac{\partial R\left(\phi ;f\right)}{\partial h\left(\alpha \right)}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,M_D;$$

(129)

$$\frac{\partial R\left(\phi ;\alpha \right)}{\partial {\sigma }_t^{\left(D,i\right)}}=\frac{\partial R\left(\phi ;f\right)}{\partial h\left(\alpha \right)}\frac{\partial h\left(\alpha \right)}{\partial {\sigma }_t^{\left(D,i\right)}}=N^{\left(D,i\right)}\frac{\partial R\left(\phi ;f\right)}{\partial h\left(\alpha \right)}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,M_D.$$

(130)

It can be readily verified that the expressions shown in Equations (122)–(130) are identical to the expressions of the corresponding sensitivities to the model parameters obtained using the 1st-CASAM in Equations (90)–(98).

It is important to note that while there are a total number of $TP=MQ+2MA+2MB+2MD+2$ first-order sensitivities of the response with respect to the primary model parameters, there are only $TF\triangleq 3+2=5$ first-order sensitivities of the response with respect to the five components of the “feature” function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$, of which three of these sensitivities are with respect to the bona-fide feature functions $g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right)$, while the other two sensitivities are with respect to the boundary parameters $\left(a,b\right)$, which are “primary parameters” in this illustrative example. The decisive advantages of the 1st-FASAM framework over the 1st-CASAM framework will become apparent when computing the second-order sensitivities, since, as will be shown in the next section, only $TF=5$ second-level adjoint sensitivity functions would be needed within the 2nd-FASAM framework, versus the much larger number of $TP=MQ+2MA+2MB+2MD+2$ second-level adjoint sensitivity functions that would be needed within the 2nd-CASAM framework.

The illustrative paradigm model analyzed in the foregoing part admitted feature functions which depend on mutually exclusive groups of primary parameters, so that none of the components of the “feature” function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$ had any primary parameters in common. It is advantageous to choose components of the “feature” function in this way (i.e., to not depend on any common primary parameters), since it is subsequently easier to compute the sensitivities with respect to the primary parameters from the sensitivities with respect to the components of the “feature” function. Of course, it may not always be possible to choose the components of the “feature” function in order not to share dependencies on the mutually independent primary model parameters. Hence, it may happen that some components of the “feature” function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$ would share dependencies on some common primary parameters. In such instances, particular attention would need to be given to ensuring that the sensitivities of the response with respect to the primary parameters are correctly extracted from the sensitivities of the response with respect to the components of the “feature” function, to ensure that all contributions are properly considered.

5. Introducing the 2nd-FASAM: Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology

The “Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (2nd-FASAM) determines the 2nd-order sensitivities ${\partial }^{2}R\left[u\left(x\right);f\left(\alpha \right)\right]/\partial f_{j_2}\partial f_{j_1}$ of the response with respect to the components of the “feature” function $f\left(\alpha \right)\triangleq {\left[g\left(\alpha \right);h\left(\alpha \right);\lambda \left(\alpha \right);\omega \left(\alpha \right)\right]}^{†}$. Conceptually, the first-order sensitivities $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]\triangleq \partial R\left[u\left(x\right);f\left(\alpha \right)\right]/\partial f_{j_1}$, which were obtained in Equation (47), are treated as “model responses” and the 2nd-order sensitivities are obtained as the “1st-order sensitivities of the 1st-order sensitivities” by applying the concepts underlying 1st-FASAM to the “response” $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]$.

Thus, the first-order G-differential of a first-order sensitivity $R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\left(\alpha \right)\right]$, $j_1=\hspace{0.17em}1,\dots ,TF$, is given by the following expression:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f;v^{\left(1\right)}\left(x\right);\delta a^{\left(1\right)}\left(x\right);\delta f\right]\right\}}_{{\alpha }^0}\\ \triangleq {\left\{\frac{d}{d\epsilon }\delta R^{\left(1\right)}\left[j_1;u\left(x\right)+\epsilon v^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right)+\epsilon \delta a^{\left(1\right)}\left(x\right);f+\epsilon \delta f\right]\right\}}_{\epsilon =0}\\ ={\displaystyle \sum _{j_2=1}^{TF}{\left\{\frac{\partial R^{\left(1\right)}\left[j_1;u;a^{\left(1\right)};f\right]}{\partial f_{j_2}}\right\}}_{{\alpha }^0}}\delta f_{j_2}+{\left\{\delta R^{\left(1\right)}\left[j_1;u;a^{\left(1\right)};f;v^{\left(1\right)}\left(x\right);\delta a^{\left(1\right)}\left(x\right)\right]\right\}}_{ind},\end{array}$$

(131)

where the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left[j_1;u;a^{\left(1\right)};f;v^{\left(1\right)}\left(x\right);\delta a^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}$ comprises all dependencies on the vectors $v^{\left(1\right)}\left(x\right)$ and $\delta a^{\left(1\right)}\left(x\right)$ of variations in the state functions $u\left(x\right)$ and $a^{\left(1\right)}\left(x\right)$, respectively, and is defined as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j_1;u;a^{\left(1\right)};f;v^{\left(1\right)};\delta a^{\left(1\right)}\right]\right\}}_{ind}\triangleq {\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}d{x}_1\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}d{x}_{TI}\left\{\frac{\partial S^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\right]}{\partial u}{v}^{\left(1\right)}\left(x\right)\right.}}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\left.+\frac{\partial S^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\right]}{\partial a^{\left(1\right)}}\delta a^{\left(1\right)}\left(x\right)\right\}}_{{\alpha }^0},\end{array}$$

(132)

The functions $v^{\left(1\right)}\left(x\right)$ and $\delta a^{\left(1\right)}\left(x\right)$ are obtained by solving the 2nd-Level Variational Sensitivity System (2nd-LVSS), which consists of concatenating the 1st-LVSS with the G-differentiated 1st-LASS, to obtain the following system (2nd-LVSS):

$${\left\{V{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]V^{\left(2\right)}\left(2;x\right)\right\}}_{{\alpha }^0}={\left\{Q_V^{\left(2\right)}\left[2;U^{\left(2\right)}\left(2;x\right);f;\delta \alpha \right]\hspace{0.17em}\right\}}_{{\alpha }^0},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x,$$

(133)

$${\left\{B_V^{\left(2\right)}\left[2;U^{\left(2\right)}\left(2;x\right);V^{\left(2\right)}\left(2;x\right);f;\delta f\right]\right\}}_{{\alpha }^0}=0\left[2\right]\triangleq {\left[0,0\right]}^{†},\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right).$$

(134)

The argument “2” which appears in the list of arguments of the vector $U^{\left(2\right)}\left(2;x\right)$ and of the “variational vector” $V^{\left(2\right)}\left(2;x\right)$ in Equation (133) indicates that each of these vectors is a two-block column vector, each block comprising a column vector of dimension $TD$, defined as follows:

$$U^{\left(2\right)}\left(2;x\right)\triangleq \left(\begin{array}{c}{u}^{\left(1\right)}\left(x\right)\\ a^{\left(1\right)}\left(x\right)\end{array}\right);\hspace{0.17em}\hspace{0.17em}{V}^{\left(2\right)}\left(2;x\right)\hspace{0.17em}\triangleq \delta U^{\left(2\right)}\left(2;x\right)\triangleq \left(\begin{array}{c}{v}^{\left(2\right)}\left(1;x\right)\\ v^{\left(2\right)}\left(2;x\right)\end{array}\right)\triangleq \left(\begin{array}{c}{v}^{\left(1\right)}\left(x\right)\\ \delta a^{\left(1\right)}\left(x\right)\end{array}\right).$$

(135)

To distinguish block vectors from block matrices, two bold capital letters have been used (and will henceforth be used) to denote block matrices, as in the case of “the second-level variational matrix” $V{M}^{\left(2\right)}\left[2\times 2;u^{\left(2\right)}\left(x\right);\alpha \right]$. The “2nd-level” is indicated by the superscript “(2)”. The argument “$2\times 2$”, which appears in the list of arguments of $V{M}^{\left(2\right)}\left[2\times 2;u^{\left(2\right)}\left(x\right);f\right]$, indicates that this matrix is a $2\times 2$-dimensional block matrix comprising four matrices, each of dimensions $TD\times TD$. In particular, the structure of this block matrix is provided below:

$$V{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]\triangleq \left(\begin{array}{cc}{N}^{\left(1\right)}& 0\\ V_{21}^{\left(2\right)}& V_{22}^{\left(2\right)}\end{array}\right),$$

(136)

where:

$$V_{21}^{\left(2\right)}\left(u;a^{\left(1\right)};f\right)\triangleq \frac{\partial \left\{{\left[N^{\left(1\right)}\left(u;f\right)\right]}^{*}{a}^{\left(1\right)}\right\}}{\partial u}-\frac{q_A^{\left(1\right)}\left[u\left(x\right);f\right]}{\partial u};$$

(137)

$$V_{22}^{\left(2\right)}\left(u;f\right)\triangleq {\left[N^{\left(1\right)}\left(u;f\right)\right]}^{*};$$

(138)

The other quantities which appear in Equations (133) and (134) are two-block vectors having the same structure as $V^{\left(2\right)}\left(2;x\right)\hspace{0.17em}$, and are defined as follows:

$$Q_V^{\left(2\right)}\left[2;U^{\left(2\right)}\left(2;x\right);f;\delta f\right]\hspace{0.17em}\triangleq \left(\begin{array}{c}{q}_V^{\left(2\right)}\left(1;U^{\left(2\right)}\left(2;x\right);f;\delta f\right)\\ q_V^{\left(2\right)}\left(2;U^{\left(2\right)}\left(2;x\right);f;\delta f\right)\end{array}\right)\hspace{0.17em};$$

(139)

$$q_V^{\left(2\right)}\left[i;U^{\left(2\right)}\left(2;x\right);f;\delta f\right]\triangleq {\displaystyle \sum _{j_2=1}^{TP}{s}_V^{\left(2\right)}\left[i;j_2;U^{\left(2\right)}\left(2;x\right);\alpha \right]\delta f_{j_2},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,2;}$$

(140)

$$s_V^{\left(2\right)}\left[1;j_2;U^{\left(2\right)}\left(2;x\right);f\right]\triangleq \frac{\partial \left[Q\left(f\right)-N\left(\hspace{0.17em}u;f\right)\right]}{\partial f_{j_2}};$$

(141)

$$s_V^{\left(2\right)}\left[2;j_2;U^{\left(2\right)}\left(2;x\right);f\right]\triangleq \frac{\partial q_A^{\left(1\right)}\left[u\left(x\right);f\right]}{\partial f_{j_2}}-\frac{\partial \left\{{\left[N^{\left(1\right)}\left(u;f\right)\right]}^{*}{a}^{\left(1\right)}\left(x\right)\right\}}{\partial f_{j_2}};$$

(142)

$$\begin{gathered} B_V^{\left(2\right)}\left[2;U^{\left(2\right)}\left(2;x\right);V^{\left(2\right)}\left(2;x\right);f;\delta f\right]\triangleq \left(\begin{array}{c}\hspace{0.17em}{b}_V^{\left(1\right)}\left(u^{\left(1\right)};f;\delta u^{\left(1\right)};\delta f\right)\\ \delta b_A^{\left(1\right)}\left[U^{\left(2\right)}\left(2;x\right);V^{\left(2\right)}\left(2;x\right);f;\delta f\right]\end{array}\right) \\ \hspace{0.17em}\triangleq \left(\begin{array}{c}\hspace{0.17em}{b}_V^{\left(2\right)}\left[1;U^{\left(2\right)}\left(2;x\right);V^{\left(2\right)}\left(2;x\right);f;\delta f\right]\\ b_V^{\left(2\right)}\left[2;U^{\left(2\right)}\left(2;x\right);V^{\left(2\right)}\left(2;x\right);f;\delta f\right]\end{array}\right); \end{gathered}$$

(143)

$$\delta b_A^{\left(1\right)}\left(u;a^{\left(1\right)};f\right)\triangleq \frac{\partial b_A^{\left(1\right)}}{\partial u}{v}^{\left(1\right)}\left(x\right)+\frac{\partial b_A^{\left(1\right)}}{\partial a^{\left(1\right)}}\delta a^{\left(1\right)}\left(x\right)+\frac{\partial b_A^{\left(1\right)}}{\partial f}\delta f.$$

(144)

The argument “2” in the expression $0\left[2\right]\triangleq {\left[0,0\right]}^{†}$ in Equation (134) indicates that this expression is a two-block column vector comprising two vectors, each of which has $TD$-components, all of which are zero-valued.

The need for solving the 2nd-LVSS is circumvented by deriving an alternative expression for the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left[j_1;u;a^{\left(1\right)};f;V^{\left(2\right)}\left(2;x\right)\hspace{0.17em}\right]\right\}}_{ind}$ defined in Equation (132), in which the function $V^{\left(2\right)}\left(2;x\right)\hspace{0.17em}$ is replaced by a 2nd-level adjoint function which is independent of variations in the model parameter and state functions. This 2nd-level adjoint function is the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS), which will be constructed by using the same principles as those employed for deriving the 1st-LASS. The 2nd-LASS is constructed in a Hilbert space, denoted as ${\mathsf{H}}_2\left({\mathsf{\Omega }}_x\right)$, which comprises as elements block vectors of the same form as $V^{\left(2\right)}\left(2;x\right)$, and is endowed with the following inner product of two vectors ${\Psi }^{(2)}\left(2;x\right)\triangleq {\left[\hspace{0.17em}{\psi }^{(2)}\left(1;x\right),{\psi }^{(2)}\left(2;x\right)\right]}^{†}\in {\mathsf{H}}_2\left({\mathsf{\Omega }}_x\right)$ and ${\Phi }^{(2)}\left(x\right)\triangleq {\left[{\phi }^{(2)}\left(1;x\right),{\phi }^{(2)}\left(2;x\right)\right]}^{†}\in {\mathsf{H}}_2\left({\mathsf{\Omega }}_x\right)$:

$${〈{\Psi }^{(2)}\left(2;x\right),{\Phi }^{(2)}\left(x\right)〉}_2\triangleq {\displaystyle \sum _{i=1}^2{〈{\psi }^{(2)}\left(i;x\right),{\phi }^{(2)}\left(i;x\right)〉}_1}.$$

(145)

The inner product defined in Equation (145) will be used to construct the following 2nd-Level Adjoint Sensitivity System (2nd-LASS) for the 2nd-level adjoint function $A^{(2)}\left(2;j_1;x\right)\triangleq {\left[a^{(2)}\left(1;j_1;x\right),a^{(2)}\left(2;j_1;x\right)\right]}^{†}\in {\mathsf{H}}_2\left({\mathsf{\Omega }}_x\right)$, for each $j_1=1,\dots ,TF$:

$$\begin{array}{l}{\left\{A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]A^{(2)}\left(2;j_1;x\right)\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left\{Q_A^{\left(2\right)}\left[2;j_1;U^{\left(2\right)}\left(2;x\right);f\right]\right\}}_{{\alpha }^0},\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TF;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x,\end{array}$$

(146)

subject to boundary conditions represented as follows:

$${\left\{B_A^{\left(2\right)}\left[2;U^{\left(2\right)}\left(2;x\right);A^{(2)}\left(2;j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}=0\left[2\right];\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TF;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(147)

where

$$\begin{array}{l}{Q}_A^{\left(2\right)}\left[2;j_1;U^{\left(2\right)}\left(2;x\right);f\right]\triangleq \left(\begin{array}{c}{q}_A^{\left(2\right)}\left(1;j_1;U^{\left(2\right)};f\right)\\ q_A^{\left(2\right)}\left(2;j_1;U^{\left(2\right)};f\right)\end{array}\right)\\ \triangleq \left(\begin{array}{c}\partial S^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f;v^{\left(1\right)}\left(x\right)\right]/\partial u\\ \partial S^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f;v^{\left(1\right)}\left(x\right)\right]/\partial a^{\left(1\right)}\end{array}\right),\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TF.\end{array}$$

(148)

$$A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]\triangleq {\left[V{M}^{\left(2\right)}\left(2\times 2;U^{\left(2\right)}\left(2;x\right);f\right)\right]}^{*}=\left(\begin{array}{cc}{\left\{{\left[N^{\left(1\right)}\right]}^{*}\right\}}^{†}& {\left[V_{21}^{\left(2\right)*}\right]}^{†}\\ 0& {\left[V_{22}^{\left(2\right)*}\right]}^{†}\end{array}\right).$$

(149)

The two components $a^{(2)}\left(1;j_1;x\right)$ and $a^{(2)}\left(2;j_1;x\right)$ of the 2nd-level adjoint function $A^{(2)}\left(2;j_1;x\right)\triangleq {\left[a^{(2)}\left(1;j_1;x\right),a^{(2)}\left(2;j_1;x\right)\right]}^{†}$ are distinguished from each other by the use of the numbers “1” and “2”, respectively, in the respective list of arguments. The matrix $A{M}^{\left(2\right)}\left[2\times 2;u^{\left(2\right)}\left(x\right);f\right]$ comprises $\left(2\times 2\right)$ block matrices, each of dimensions $T{D}^2$, thus comprising a total of $\left(2\times 2\right)T{D}^2$ components (or elements). This matrix is obtained from the following relation:

$$\begin{array}{l}{\left\{{〈A^{(2)}\left(2;x\right),V{M}^{\left(2\right)}{V}^{\left(2\right)}\left(2;x\right)〉}_2\right\}}_{{\alpha }^0}={\left\{{\left[P^{\left(2\right)}\left(U^{\left(2\right)};A^{(2)};V^{\left(2\right)};f\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}\\ +{\left\{{〈V^{\left(2\right)}\left(2;x\right),A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);\alpha \right]A^{(2)}\left(2;x\right)〉}_2\right\}}_{{\alpha }^0},\end{array}$$

(150)

where the quantity ${\left\{{\left[P^{\left(2\right)}\left(U^{\left(2\right)};A^{(2)};V^{\left(2\right)};f\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}$ denotes the corresponding bilinear concomitant with the domain’s boundary, evaluated at the nominal values for the parameters and respective state functions. The 2nd-level adjoint boundary/initial conditions represented by Equation (147) are determined by requiring that:

(a)

they must be independent of unknown values of $V^{\left(2\right)}\left(2;x\right)$;

(b)

the substitution of the boundary and/or initial conditions represented by Equations (147) and (134) into the expression of ${\left\{{\left[P^{\left(2\right)}\left(U^{\left(2\right)};A^{(2)};V^{\left(2\right)};f\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}$ must cause all terms containing unknown values of $V^{\left(2\right)}\left(2;x\right)$ to vanish.

Using in Equation (132) the relations defining 2nd-LASS together with the 2nd-LVSS and the relation provided in Equation (150) yields the following alternative expression for the indirect-effect term in terms of the 2nd-level adjoint sensitivity function $A^{(2)}\left(2;j_1;x\right)\triangleq {\left[a^{(2)}\left(1;j_1;x\right),a^{(2)}\left(2;j_1;x\right)\right]}^{†}$:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j_1;u;a^{\left(1\right)};f;V^{\left(2\right)}\left(2;x\right)\hspace{0.17em}\right]\right\}}_{ind}=-{\left\{{\left[{\widehat{P}}^{\left(2\right)}\left(U^{\left(2\right)};A^{(2)};f;\delta f\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\left\{{〈A^{(2)}\left(2;j_1;x\right),Q_V^{\left(2\right)}\left[2;U^{\left(2\right)}\left(2;x\right);f;\delta f\right]〉}_2\right\}}_{{\alpha }^0},\end{array}$$

(151)

where ${\left\{{\left[{\widehat{P}}^{\left(2\right)}\left(U^{\left(2\right)};A^{(2)};f;\delta f\right)\right]}_{\partial {\mathsf{\Omega }}_x}\right\}}_{{\alpha }^0}$ denotes residual boundary terms which may not have vanished after having used the boundary and/or initial conditions represented by Equations (134) and (147).

Substituting the expression obtained in Equation (151) into Equation (131) yields the following expression:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j_1;U^{\left(2\right)}\left(2;x\right);A^{(2)}\left(2;j_1;x\right);f;\delta f\right]\right\}}_{{\alpha }^0}\\ ={\displaystyle \sum _{j_2=1}^{TP}{\left\{R^{\left(2\right)}\left[j_2;j_1;U^{\left(2\right)}\left(2;x\right);A^{(2)}\left(2;j_1;x\right);f\right]\right\}}_{{\alpha }^0}}\delta f_{j_2},\hspace{0.17em}\hspace{0.17em}{j}_1=\hspace{0.17em}1,\dots ,TF\hspace{0.17em},\end{array}$$

(152)

where the quantity $R^{\left(2\right)}\left[j_2;j_1;U^{\left(2\right)}\left(2;x\right);A^{(2)}\left(2;j_1;x\right);f\right]$ denotes the second-order sensitivity of the generic scalar-valued response $R\left[u\left(x\right);\alpha \right]$ with respect to the components $f_{j_1}$ and $f_{j_2}$ of the “feature” function $f$, computed at the nominal values of the parameters and respective state functions, and has the following expression for $j_1,j_2=1,\dots ,TF$:

$$\begin{array}{l}{R}^{\left(2\right)}\left[j_2;j_1;U^{\left(2\right)}\left(2;x\right);A^{(2)}\left(2;j_1;x\right);f\right]\triangleq \frac{\partial R^{\left(1\right)}\left[j_1;u\left(x\right);a^{\left(1\right)}\left(x\right);f\right]}{\partial j_{j_2}}\\ +{\displaystyle \sum _{i=1}^2{〈a^{(2)}\left(i;j_1;x\right),s_V^{\left(2\right)}\left[i;j_2;U^{\left(2\right)}\left(2;x\right);f\right]〉}_1}\\ -\frac{{\left\{\partial {\widehat{P}}^{\left(2\right)}\left[U^{\left(2\right)}\left(2;x\right);A^{(2)}\left(2;j_1;x\right);f\right]\right\}}_{\partial {\mathsf{\Omega }}_x}}{\partial f_{j_2}}\triangleq \frac{{\partial }^{2}R\left[u\left(x\right);\alpha \right]}{\partial f_{j_2}\partial f_{j_1}}.\end{array}$$

(153)

Notably, the 2nd-LASS is independent of variations $\delta f$ (and, hence, of parameter variations $\delta \alpha$) and variations $V^{\left(2\right)}\left(2;x\right)$ in the respective state functions. It is also important to note that the ${\left(2\times TD\right)}^2$-dimensional matrix $A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]$ is independent of the index $j_1$. Only the source-term $Q_A^{\left(2\right)}\left[2;j_1;U^{\left(2\right)}\left(2;x\right);f\right]$ depends on the index $j_1$. Therefore, the same solver can be used to invert the matrix $A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);\hspace{0.17em}f\right]$ in order to solve numerically the 2nd-LASS for each $j_1$-dependent source $Q_A^{\left(2\right)}\left[2;j_1;U^{\left(2\right)}\left(2;x\right);f\right]$ in order to obtain the corresponding $j_1$-dependent $2\times TD$-dimensional 2nd-level adjoint function $A^{(2)}\left(2;j_1;x\right)\triangleq {\left[a^{(2)}\left(1;j_1;x\right),a^{(2)}\left(2;j_1;x\right)\right]}^{†}$. Computationally, it would be most efficient to store, if possible, the inverse matrix ${\left\{A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]\right\}}^{-1}$, in order to multiply directly the inverse matrix ${\left\{A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]\right\}}^{-1}$ with the corresponding source term $Q_A^{\left(2\right)}\left[2;j_1;U^{\left(2\right)}\left(2;x\right);f\right]$ for each index $j_1$, in order to obtain the corresponding $j_1$-dependent $2\times TD$-dimensional 2nd-level adjoint function $A^{(2)}\left(2;j_1;x\right)\triangleq {\left[a^{(2)}\left(1;j_1;x\right),a^{(2)}\left(2;j_1;x\right)\right]}^{†}$.

Since the adjoint matrix $A{M}^{\left(2\right)}\left[2\times 2;U^{\left(2\right)}\left(2;x\right);f\right]$ is block-diagonal, solving the 2nd-LASS is equivalent to solving two 1st-LASSs, with two different source terms. Thus, the “solvers” and the computer program used for solving the 1st-LASS can also be used for solving the 2nd-LASS. The 2nd-LASS was designated as the “second-level” rather than the “second-order” adjoint sensitivity system, since the 2nd-LASS does not involve any explicit 2nd-order G-derivatives of the operators underlying the original system, but involves the inversion of the same operators that need to be inverted for solving the 1st-LASS.

If the 2nd-LASS is solved $TF$ times, the 2nd-order mixed sensitivities $R^{\left(2\right)}\left[j_2;j_1;U^{\left(2\right)}\left(2;x\right);A^{(2)}\left(2;j_1;x\right);f\right]\equiv {\partial }^{2}R/\partial f_{j_2}\partial f_{j_1}$ will be computed twice, in two different ways, in terms of two distinct 2nd-level adjoint functions. Consequently, the symmetry property ${\partial }^{2}R\left[u\left(x\right);f\right]/\partial f_{j_2}\partial f_{j_1}={\partial }^{2}R\left[u\left(x\right);f\right]/\partial f_{j_1}\partial f_{j_2}$ provides an intrinsic (numerical) verification that the 1st-level adjoint function $a^{(1)}\left(x\right)$ and the components of the 2nd-level adjoint function $A^{(2)}\left(2;j_1;x\right)$ are computed accurately.

Similarities and Differences between the 2nd-FASAM and the 2nd-CASAM

As has been discussed in Section 3.1, the “First-Order Comprehensive Adjoint Sensitivity Analysis Methodology” (1st-CASAM) developed by Cacuci [7,8] computes the exactly derived expressions of the first-order sensitivities of a model response directly, with respect to the model’s parameters. In contradistinction, the 1st-FASAM presented in Section 3 computes the exactly derived expressions of the first-order sensitivities of a model response with respect to functions/features of model. Mathematically, the formal correspondence between the 1st-FASAM and the 1st-CASAM is obtained by replacing the components of the “features” vector-valued function $f\left(\alpha \right)\triangleq {\left[g\left(\alpha \right);h\left(\alpha \right);\lambda \left(\alpha \right);\omega \left(\alpha \right)\right]}^{†}\triangleq {\left[f_1\left(\alpha \right),\dots ,f_{TF}\left(\alpha \right)\right]}^{†}$ within the 1st-FASAM formalism with the components of the vector of primary parameters $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}$, which would yield the 1st-CASAM mathematical formalism.

The formal correspondence described above between the 1st-FASAM and the 1st-CASAM also translates, for each of the respective first-order sensitivities, to the formal correspondence between the 2nd-FASAM and the 2nd-CASAM frameworks; i.e., replacing formally the components of $f\left(\alpha \right)\triangleq {\left[f_1\left(\alpha \right),\dots ,f_{TF}\left(\alpha \right)\right]}^{†}$ in the 2nd-FASAM framework with the components of $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}$ yields the 2nd-CASAM framework. It is very important to note, however, that while there are a total number of $TP$ primary model parameters (and, hence, $TP$ first-order sensitivities) within the 1st-CASAM, there are only $TF$ feature functions (and, hence, $TF$ first-order sensitivities), within the 1st-FASAM, where $TF\ll TP$. Each of the respective first-order sensitivity becomes a “response” that generates a corresponding “second-level adjoint sensitivity system” (2nd-LASS) for computing the corresponding second-order sensitivities. Therefore, there are a total number of $TP$ “second-level adjoint sensitivity systems” to be solved within the 2nd-CASAM framework, but there are only $TF$ “second-level adjoint sensitivity systems” to be solved within the 2nd-FASAM formalism. Since the 2nd-LASS within either the 2nd-CASAM or the 2nd-CASAM frameworks involves the same operators on their respective left-hand sides (only the “sources” on the respective right-hand sides are different), it follows that the same computational effort is required to solve any of these 2nd-LASS. Thus, since $TF\ll TP$, there are far fewer ($TF$) large-scale computations needed to solve the 2nd-LASS for obtaining the second-order sensitivities of a response with respect to the components of the “feature” function $f\left(\alpha \right)\triangleq {\left[f_1\left(\alpha \right),\dots ,f_{TF}\left(\alpha \right)\right]}^{†}$ within the 2nd-FASAM framework, in comparison to the considerably larger number ($TP$) of large-scale computations needed to solve the 2nd-LASS for obtaining the second-order sensitivities of a response with respect to the components of $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}$ within the 2nd-CASAM framework.

Performing $TP$ computations using the 2nd-LASS within the 2nd-CASAM methodology, each such computation corresponding to one of the $TP$ first-order sensitivities computed within the 1st-CASAM will yield a total of $TP\times TP$ second-order response sensitivities with respect to the primary model parameters $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}$, thereby computing the mixed second-order sensitivities twice, using distinct second-level adjoint sensitivity functions. This property of the 2nd-CASAM provides an intrinsic mechanism for verifying the accuracy of the adjoint sensitivity functions produced by solving the corresponding 2nd-LASS.

The above intrinsic verification mechanism, which ensures that the second-level adjoint sensitivity functions are computed accurately, is also inherent to the 2nd-FASAM. Solving the 2nd-LASS corresponding to each of the $TF$ first-order sensitivities computed within the 1st-FASAM will yield a total of $TF\times TF$ second-order response sensitivities with respect to the components of the feature function $f\left(\alpha \right)\triangleq {\left[f_1\left(\alpha \right),\dots ,f_{TF}\left(\alpha \right)\right]}^{†}$, thereby computing the mixed second-order sensitivities twice, using distinct second-level adjoint sensitivity functions. Due to the symmetry of the mixed second-order sensitivities, the two distinct solutions of the respective 2nd-LASS must produce the same numerical result for the second-order sensitivity under consideration. These considerations will be explicitly illustrated in the application to the paradigm particle transport model considered in Section 6, below.

After applying the 1st-FASAM and the 2nd-FASAM to obtain the first-order and, respectively, second-order sensitivities of a response $R\left[u\left(x\right);\alpha \right]$ with respect to the components of the “feature” function $f\left(\alpha \right)\triangleq {\left[f_1\left(\alpha \right),\dots ,f_{TF}\left(\alpha \right)\right]}^{†}$, the computation of the second-order sensitivities of the response $R\left[u\left(x\right);\alpha \right]$ with respect to the components of the vector $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}$ of the primary model parameters is obtained at practically no additional computational costs, by simply using the “chain rule” in conjunction with Equation (57) to obtain:

$$\hspace{1em}\frac{{\partial }^{2}R\left[u\left(x\right);\alpha \right]}{\partial {\alpha }_{j_2}\partial {\alpha }_{j_1}}=\hspace{1em}\hspace{0.17em}\frac{\partial }{\partial {\alpha }_{j_2}}\left\{{\displaystyle \sum _{i=1}^{TF}\hspace{1em}\frac{\partial R\left[u\left(x\right);\alpha \right]}{\partial f_i}}\frac{\partial f_i\left(\alpha \right)}{\partial {\alpha }_{j_1}}\right\},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}{j}_2=1,\dots ,j_1.$$

(154)

6. Applying the 2nd-FASAM to Obtain the Second-Order Response Sensitivities to Functions/Features of Parameters for the Paradigm Particle Transport Model Considered in Section 4

This section illustrates the application of the 2nd-FASAM to obtain the exact expressions of the second-order sensitivities of the “kerma” response [11] with respect to the components of the “feature” function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$ for the paradigm particle transport model introduced in Section 4. As has been shown in Section 5, the 2nd-FASAM obtains the second-order sensitivities by computing “the first-order sensitivities of the first-order sensitivities.” Since the “feature” function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$ comprises $TF=5$ components, there are five first-order sensitivities $R^{\left(1\right)}\left[j_1;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left[u\left(x\right);f\right]/\partial f_{j_1},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TF=5$, which were obtained in Equations (119)–(123). The second-order sensitivities ${\partial }^{2}R\left[u\left(x\right);f\right]/\partial f_{j_1}\partial f_{j_2}$, $j_1,\hspace{0.17em}{j}_2=1,\dots ,\hspace{0.17em}TF=5$ of the model response with respect to the components of $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$ will be determined by applying the 2nd-FASAM to the first-order sensitivities $R^{\left(1\right)}\left[j_1;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]$. As has been shown in Section 5, above, for each first-order sensitivity, there will correspond a “second-level adjoint sensitivity function” which will be used for obtaining the second-order sensitivities that arise from the respective first-order sensitivity. In general, a first-order sensitivity will depend on both the original forward function and on the corresponding adjoint function; in general, therefore, a “second-level adjoint sensitivity function” will be a two-component function, having one component corresponding to the variation in the original forward function and having the second component corresponding to the variation in the first-level adjoint sensitivity function. For the illustrative particle transport model considered in Section 5, however, an examination of Equations (119)–(123) reveals the following characteristics:

(i)

The first-order sensitivity $R^{\left(1\right)}\left[j_1=1;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$ depends only on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from $R^{\left(1\right)}\left[j_1=1;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$ will be obtained in terms of a “two-component second-level adjoint sensitivity function” $A^{(2)}\left(2;j_1=1;x\right)\triangleq {\left[a^{(2)}\left(1;j_1=1;z,\omega \right)\equiv 0,\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;j_1=1;z,\omega \right)\right]}^{†}$ which will have a single non-zero component, $a^{(2)}\left(2;j_1=1;z,\omega \right)$, corresponding to $\delta a^{\left(1\right)}$.

(ii)

The first-order sensitivity $R^{\left(1\right)}\left[j_1=2;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_2\left(\alpha \right)$ is the only first-order sensitivity that depends on both the forward function $\phi \left(z,\omega \right)$ and on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. Consequently, when applying the 2nd-FASAM, the second-order sensitivities which will stem from $R^{\left(1\right)}\left[j_1=2;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_2\left(\alpha \right)$ will be obtained in terms of a “two-component second-level adjoint sensitivity function” $A^{(2)}\left(2;j_1=2;x\right)\triangleq {\left[a^{(2)}\left(1;j_1=2;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;j_1=2;z,\omega \right)\right]}^{†}$ which will have two non-zero components: one component corresponding to $\delta \phi$, and the second component corresponding to $\delta a^{\left(1\right)}$.

(iii)

The first-order sensitivity $R^{\left(1\right)}\left[j_1=3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h\left(\alpha \right)$ depends only on the forward function,$\phi \left(z,\omega \right)$ but does not depend on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from $R^{\left(1\right)}\left[j_1=3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h\left(\alpha \right)$ will be obtained in terms of a “two-component second-level adjoint sensitivity function” $A^{(2)}\left(2;j_1=3;x\right)\triangleq {\left[a^{(2)}\left(1;j_1=3;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;j_1=3;z,\omega \right)\equiv 0\right]}^{†}$, which will have a single non-zero component corresponding to $\delta \phi$.

(iv)

The first-order sensitivity $R^{\left(1\right)}\left[j_1=4;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial a$ depends only on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from $R^{\left(1\right)}\left[j_1=4;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial a$ will be obtained in terms of a “two-component second-level adjoint sensitivity function” $A^{(2)}\left(2;j_1=4;x\right)\triangleq {\left[a^{(2)}\left(1;j_1=4;z,\omega \right)\equiv 0,\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;j_1=4;z,\omega \right)\right]}^{†}$, which will have a single non-zero component corresponding to $\delta a^{\left(1\right)}$. This situation is similar to the determination of the second-order sensitivities stemming from $R^{\left(1\right)}\left[j_1=1;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$, but the respective second-order adjoint sensitivity functions will differ from each other because, although the respective 2nd-Level Adjoint Sensitivity Systems (2nd-LASS) will have the same left-hand side, the respective 2nd-LASS will have distinct right-hand sides (sources).

(v)

The first-order sensitivity $R^{\left(1\right)}\left[j_1=5;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial b$ depends only on the forward function $\phi \left(z,\omega \right)$; it does not depend on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. This situation is similar to the above “case (iii)” for $R^{\left(1\right)}\left[j_1=3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h\left(\alpha \right)$. Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from $R^{\left(1\right)}\left[j_1=5;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial b$ will be obtained in terms of a “two-component second-level adjoint sensitivity function” $A^{(2)}\left(2;j_1=5;x\right)\triangleq {\left[a^{(2)}\left(1;j_1=5;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;j_1=5;z,\omega \right)\equiv 0\right]}^{†}$, which will have a single non-zero component corresponding to $\delta \phi$. Again, the 2nd-LASS corresponding to $R^{\left(1\right)}\left[j_1=5;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial b$ and $R^{\left(1\right)}\left[j_1=3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h\left(\alpha \right)$, respectively, will have the same left-hand sides but different right-hand sides (sources).

The above considerations will be detailed in the following five subsections, below.

6.1. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivity $\partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$

The 1st-order sensitivity $\partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$ is defined in Equation (119), which is recast into the following “response-like” form:

$$R^{\left(1\right)}\left[1;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \frac{\partial R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}\hspace{0.17em}\triangleq R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]$$

(155)

As has been noted by defining the last term in Equation (155), the 1st-order sensitivity $R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$ does not depend on the original forward function $\phi \left(z,\omega \right)$ (i.e., the particle flux), but only depends on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. The digit “1” in the list of arguments of $R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]$ indicates that $R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]$ is the “first to be considered” of the five first-order sensitivities of the model response with respect to the five components of the “feature” function $f\left(\alpha \right)$, which were obtained in Equations (119)–(123).

According to the principles of the 2nd-CASAM presented in Section 5, above, the second-order sensitivities which stem from $\partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$ are provided by the first-order G-differential of $R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$. Applying the definition of the G-differential to Equation (155) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right);\delta a^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}={\left\{\frac{d}{d\epsilon }{\left[{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega {\displaystyle \underset{a+\epsilon \delta a}{\overset{b+\epsilon \delta b}{\int }}\left(a^{\left(1\right)}+\epsilon \delta a^{\left(1\right)}\right)\delta \left(z-a-\epsilon \delta a\right)dz}}\hspace{0.17em}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\hspace{0.17em}\\ \hspace{0.17em}={\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(156)

where the direct-effect term ${\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}$ depends only on variations $\delta f$, while the indirect-effect term ${\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ depends only on variations $\delta a^{\left(1\right)}$; these terms are defined, respectively, as follows:

$${\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega {\left[\frac{d{a}^{\left(1\right)}\left(z,\omega \right)}{dz}\right]}_{z=a}\left(\delta a\right)}\right\}}_{{\alpha }^0}\hspace{0.17em},$$

(157)

$${\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\triangleq {\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\omega \hspace{0.17em}\delta a^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}\right\}}_{{\alpha }^0}$$

(158)

The direct-effect term defined by Equation (157) can be evaluated at this stage by using Equation (116) to obtain:

$${\left[\frac{d{a}^{\left(1\right)}\left(z,\omega \right)}{dz}\right]}_{z=a}=\frac{h\left(\alpha \right)g_2\left(\alpha \right)}{{\omega }^2}\exp \left[\frac{g_2\left(\alpha \right)\left(a-b\right)}{\omega }\right]$$

(159)

Inserting the result obtained in Equation (159) into Equation (157) yields:

$${\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq \left(\delta a\right){\left\{h\left(\alpha \right)g_2\left(\alpha \right)E_1\left[g_2\left(\alpha \right)\left(a-b\right)\right]\right\}}_{{\alpha }^0}\hspace{0.17em}$$

(160)

The indirect-effect term defined in Equation (158) can be evaluated only after having determined the variational function $\delta a^{\left(1\right)}\left(z,\omega \right)$, which is the solution of the G-differential of the 1st-LASS, which is, in turn, obtained by applying the definition of the (first-order) G-differential to Equations (114) and (115), i.e.:

$$\begin{array}{l}{\left\{\frac{d}{d\epsilon }{\left[-\omega \frac{d\left(a^{\left(1\right)}+\epsilon \delta a^{\left(1\right)}\right)}{dz}+\left(g_2+\epsilon \delta g_2\right)\left(a^{\left(1\right)}+\epsilon \delta a^{\left(1\right)}\right)\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\\ ={\left\{\frac{d}{d\epsilon }{\left[\left(h+\epsilon \delta h\right)\delta \left(z-b-\epsilon \delta b\right)\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\hspace{0.17em},\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1,\end{array}$$

(161)

$${\left\{\frac{d}{d\epsilon }\left[a^{\left(1\right)}\left(z=b^0+\epsilon \delta b;\omega \right)+\epsilon \delta a^{\left(1\right)}\left(z=b^0+\epsilon \delta b;\omega \right)\right]\right\}}_{\epsilon =0}=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(162)

Carrying out the operations indicated in Equations (161) and (162) yields the following 2nd-Level Forward Variational System (2nd-LVSS) for the function $\delta a^{\left(1\right)}\left(z,\omega \right)$:

$$\begin{array}{l}{\left\{-\omega \frac{d}{dz}\delta a^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)\delta a^{\left(1\right)}\left(z,\omega \right)\right\}}_{{\alpha }^0}\\ ={\left\{-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right\}}_{{\alpha }^0},\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1,\end{array}$$

(163)

$${\left\{{\left[\delta a^{\left(1\right)}\left(z,\omega \right)\right]}_{z=b}\right\}}_{{\alpha }^0}=-{\left\{{\left[\frac{d{a}^{\left(1\right)}\left(z,\omega \right)}{dz}\right]}_{z=b}\right\}}_{{\alpha }^0}=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(164)

In principle, the 2nd-LVSS could be solved to obtain $\delta a^{\left(1\right)}\left(z,\omega \right)$ for every possible variation in $\delta g_2$, $\delta h$ and/or $\delta b$, which would be prohibitively expensive and also unnecessary. Instead, solving the 2nd-LVSS is avoided altogether by recasting the expression of the indirect-effect term by using a second-level adjoint function, which will be the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS) to be constructed by applying the general 2nd-FASAM as outlined in Section 5, above. As has been discussed in the foregoing, the indirect-effect term ${\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$, defined in Equation (158), only involves the variational function $\delta a^{\left(1\right)}\left(z,\omega \right)$, but does not also involve the variational function $\delta \phi \left(z,\omega \right)$. Consequently, the 2nd-level adjoint sensitivity function will comprise a single non-zero component (which will correspond to $\delta a^{\left(1\right)}$) rather than two non-zero components—as would have been the case if the first-order sensitivity had depended on both the flux $\phi \left(z,\omega \right)$ and the 1st-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. Hence, the inner product appropriate for recasting ${\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ in terms of a second-level adjoint sensitivity function is the same as that defined in Equation (80). Thus, introducing a one-component 2nd-level adjoint sensitivity function (which is as yet undefined), denoted as $a^{(2)}\left(2;1;z,\omega \right)$, and forming the inner product of this function with Equation (163), yields the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(2;1;z,\omega \right)\left[-\omega \frac{d}{dz}\delta a^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)\delta a^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}=\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }\right.{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)\right.}\\ {\left.\left.\hspace{1em}\hspace{1em}+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(165)

Note that the first argument of $a^{(2)}\left(2;1;z,\omega \right)$ indicates that only the “2nd-component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in $a^{(2)}\left(2;1;z,\omega \right)$ refers to the index/label “$j_1=1$,”which indicates that this second-level adjoint sensitivity function will correspond to “the first considered” first-order sensitivity.

Integrating by parts the left side of Equation (165) over the independent variable z yields the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(2;1;z,\omega \right)\left[-\omega \frac{d}{dz}\delta a^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)\delta a^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }{\left\{\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(2;1;a,\omega \right)\delta a^{\left(1\right)}\left(a,\omega \right)-a^{(2)}\left(2;1;b,\omega \right)\delta a^{\left(1\right)}\left(b,\omega \right)\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\delta a^{\left(1\right)}\left(z,\omega \right)\left[\omega \frac{d}{dz}{a}^{(2)}\left(2;1;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;1;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}.\end{array}$$

(166)

The last term on the right side of Equation (166) is now required to represent the indirect-effect term defined in Equation (158) by imposing the following relationship:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\delta a^{\left(1\right)}\left(z,\omega \right)\left[\omega \frac{d}{dz}{a}^{(2)}\left(2;1;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;1;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\hspace{0.17em}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\omega \hspace{0.17em}\delta a^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}\right\}}_{{\alpha }^0}.\end{array}$$

(167)

The relation in Equation (167) implies that the following equation holds, in the weak sense, at the nominal parameter values:

$${\left\{\omega \frac{d}{dz}{a}^{(2)}\left(2;1;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;1;z,\omega \right)\right\}}_{{\alpha }^0}={\left\{\omega \delta \left(z-a\right)\right\}}_{{\alpha }^0}\hspace{0.17em},\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(168)

The definition of the function $a^{(2)}\left(2;1;z,\omega \right)$ is now completed by requiring that the unknown value of the function $\delta a^{\left(1\right)}\left(a,\omega \right)$ be eliminated from appearing on the right side of Equation (168). This requirement is met by imposing the following condition to be satisfied by the function $a^{(2)}\left(2;1;z,\omega \right)$ on the model’s boundary:

$$a^{(2)}\left(2;1;z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=a^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(169)

Altogether, Equations (168) and (169) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS), for determining the second-level adjoint sensitivity function $a^{(2)}\left(2;1;z,\omega \right)$. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain $a^{(2)}\left(2;1;z,\omega \right)$ For the paradigm model under consideration, the 1st-LASS given by Equations (168) and (169) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function $a^{(2)}\left(2;1;z,\omega \right)$:

$$a^{(2)}\left(2;1;z,\omega \right)=H\left(z-a\right)\exp \left[\frac{g_2\left(\alpha \right)\left(a-z\right)}{\omega }\right]$$

(170)

Collecting the results obtained in Equations (163)–(170) leads to the following expression for the indirect-effect term:

$$\begin{array}{l}\hspace{0.17em}{\left\{\delta R^{\left(1\right)}{\left(1;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}=\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }\right.{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)}\\ {\left.\left.\times \hspace{0.17em}\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)\right.+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]\right\}}_{{\alpha }^0}\\ ={\left\{\delta R^{\left(1\right)}{\left[1;a^{\left(1\right)};f;a^{(2)}\left(2;1;z,\omega \right)\right]}_{{\alpha }^0}\right\}}_{ind}.\end{array}$$

(171)

Note that the dependence of ${\left\{\delta R^{\left(1\right)}{\left[1;a^{\left(1\right)};f;a^{(2)}\left(2;1;z,\omega \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ on $\delta a^{\left(1\right)}$ has been replaced in the last equality in Equation (171) by the dependence on $a^{(2)}\left(2;1;z,\omega \right)$.

Adding the results obtained in Equations (171) and (160) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right);a^{(2)}\left(2;1;z,\omega \right);\delta f\right]\right\}}_{{\alpha }^0}=\left(\delta a\right){\left\{h\left(\alpha \right)g_2\left(\alpha \right)E_1\left[g_2\left(\alpha \right)\left(a-b\right)\right]\right\}}_{{\alpha }^0}\hspace{0.17em}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)\left.\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)\right.+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]}\right\}}_{{\alpha }^0}.\end{array}$$

(172)

Note that the dependence of $\delta R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right);a^{(2)}\left(2;1;z,\omega \right);\delta f\right]$ on the variational function $\delta a^{\left(1\right)}$ has been replaced in Equation (172) by the dependence on the second-level adjoint sensitivity function $a^{(2)}\left(2;1;z,\omega \right)$. Identifying the expressions that multiply the respective components of $\delta f\left(\alpha \right)\triangleq {\left[\delta g_1\left(\alpha \right),\delta g_2\left(\alpha \right);\delta h\left(\alpha \right);\delta a;\delta b\right]}^{†}$, and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (170), yields the following results for the second-order sensitivities that stem from the first-order sensitivity $R^{\left(1\right)}\left[1;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_1\left(\alpha \right)$:

$${\left\{\frac{\partial R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]}{\partial g_1\left(\alpha \right)}\right\}}_{{\alpha }^0}={\left\{\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)\partial g_1\left(\alpha \right)}\right\}}_{{\alpha }^0}\hspace{0.17em}=0$$

(173)

$$\begin{array}{l}{\left\{\frac{\partial R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]}{\partial g_2\left(\alpha \right)}\right\}}_{{\alpha }^0}={\left\{\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)\partial g_1\left(\alpha \right)}\right\}}_{{\alpha }^0}\hspace{0.17em}\hspace{0.17em}\\ =-{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)a^{\left(1\right)}\left(z,\omega \right)}}\right\}}_{{\alpha }^0}=-{\left\{h\left(\alpha \right)\left(b-a\right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}\end{array}$$

(174)

$$\begin{array}{l}\hspace{0.17em}{\left\{\frac{\partial R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]}{\partial h\left(\alpha \right)}\right\}}_{{\alpha }^0}={\left\{\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial g_1\left(\alpha \right)}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ ={\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)\delta \left(z-b\right)}\right\}}_{{\alpha }^0}={\left\{E_2\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}\end{array}$$

(175)

$$\begin{array}{l}{\left\{\frac{\partial R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]}{\partial a}\right\}}_{{\alpha }^0}={\left\{\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial g_1\left(\alpha \right)}\right\}}_{{\alpha }^0}=\hspace{0.17em}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega {\left[\frac{d{a}^{\left(1\right)}\left(z,\omega \right)}{dz}\right]}_{z=a}}\right\}}_{{\alpha }^0}\\ ={\left\{h\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}\hspace{0.17em}\end{array}$$

(176)

$$\begin{array}{l}{\left\{\frac{\partial R^{\left(1\right)}\left[1;a^{\left(1\right)};f\left(\alpha \right)\right]}{\partial b}\right\}}_{{\alpha }^0}={\left\{\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial g_1\left(\alpha \right)}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}=-{\left\{h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)}\hspace{0.17em}{\delta }^{\prime }\left(z-b\right)\right\}}_{{\alpha }^0}\hspace{0.17em}=-{\left\{h\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(177)

The validity of the expressions obtained in Equations (173)–(177) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).

6.2. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities $\partial R\left(\phi ;f\right)/\partial g_2\left(\alpha \right)$

The 1st-order sensitivity $\partial R\left(\phi ;f\right)/\partial g_2\left(\alpha \right)$ is defined in Equation (120), which is recast into the following “response-like” form:

$$R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \frac{\partial R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)}=-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)dz}$$

(178)

The digit “2” in the list of arguments of $R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]$ indicates that this is the “2nd first-order sensitivity to be considered” of the five first-order sensitivities which were obtained in Equations (119)–(123). As indicated in Equation (178), the first-order sensitivity $R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]$ depends on both the original forward function $\phi \left(z,\omega \right)$ and also on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$.

According to the principles of the 2nd-CASAM presented in Section 5, above, the second-order sensitivities which stem from $\partial R\left(\phi ;f\right)/\partial g_2\left(\alpha \right)$ are provided by the first-order G-differential of $R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_2\left(\alpha \right)$. Applying the definition of the G-differential to Equation (178) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f;\delta a^{\left(1\right)};v^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}=-{\left\{\frac{d}{d\epsilon }{\left[{\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega {\displaystyle \underset{a+\epsilon \delta a}{\overset{b+\epsilon \delta b}{\int }}\left(a^{\left(1\right)}+\epsilon \delta a^{\left(1\right)}\right)\hspace{0.17em}\left(\phi +\epsilon v^{\left(1\right)}\right)dz}}\hspace{0.17em}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\hspace{0.17em}\\ \hspace{0.17em}={\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta a^{\left(1\right)};v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(179)

where the direct-effect term ${\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}$ depends only on variations $\delta f$, while the indirect-effect term ${\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta a^{\left(1\right)};v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ depends only on variations $\delta a^{\left(1\right)}$ and $v^{\left(1\right)}$; these two terms are defined, respectively, as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq -{\left\{\left(\delta b\right){\displaystyle \underset{0}{\overset{1}{\int }}{a}^{\left(1\right)}\left(b,\omega \right)\phi \left(b,\omega \right)d\omega \hspace{0.17em}}\right\}}_{{\alpha }^0}\\ +{\left\{\left(\delta a\right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}{a}^{\left(1\right)}\left(a,\omega \right)\phi \left(a,\omega \right)d\omega \hspace{0.17em}}\right\}}_{{\alpha }^0},\end{array}$$

(180)

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta a^{\left(1\right)};v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\\ \triangleq -{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\left[a^{\left(1\right)}\left(z,\omega \right)v^{\left(1\right)}\left(z,\omega \right)+\delta a^{\left(1\right)}\left(z,\omega \right)\phi \left(z,\omega \right)\right]dz}}\right\}}_{{\alpha }^0}.\end{array}$$

(181)

The direct-effect term defined by Equation (180) can be evaluated at this stage by using Equations (107) and (116) to obtain:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq -\left(\delta b\right){\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega \hspace{0.17em}{a}^{\left(1\right)}\left(b,\omega \right)\phi \left(b,\omega \right)}\right\}}_{{\alpha }^0}\\ +\left(\delta a\right){\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega \hspace{0.17em}{a}^{\left(1\right)}\left(a,\omega \right)\phi \left(a,\omega \right)}\right\}}_{{\alpha }^0}=\left(\delta a\right)g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right],\end{array}$$

(182)

Note that the first term on the right-hand side of Equation (182) vanishes because of Equation (115). The indirect-effect term defined in Equation (181) can be evaluated only after having determined the variational functions $\delta a^{\left(1\right)}\left(z,\omega \right)$ and $v^{\left(1\right)}\left(z,\omega \right)$. Recall that the variation $\delta a^{\left(1\right)}\left(z,\omega \right)$ is the solution of Equations (163) and (164), while the variation $v^{\left(1\right)}\left(z,\omega \right)$ is the solution of Equations (112) and (113). The need for solving repeatedly these equations to obtain the functions $\delta a^{\left(1\right)}\left(z,\omega \right)$ and $v^{\left(1\right)}\left(z,\omega \right)$ for all possible parameter variation is circumvented by applying the 2nd-FASAM. Since the indirect-effect term depends on both $v^{\left(1\right)}\left(z,\omega \right)$ and $\delta a^{\left(1\right)}\left(z,\omega \right)$, the second-level adjoint function needed for recasting the expression of the indirect-effect term will have two non-zero components, and the inner product will therefore have the form shown in Equation (145). Thus, forming the inner product of an as yet undefined second-level function of the form $A^{(2)}\left(2;j_1=2;x\right)\triangleq {\left[a^{(2)}\left(1;j_1=2;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;j_1=2;z,\omega \right)\right]}^{†}$ with Equations (112) and (163), respectively, yields the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(1;2;z,\omega \right)\left[\omega \frac{d{v}^{\left(1\right)}\left(z,\omega \right)}{dz}+g_2\left(\alpha \right)v^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(2;2;z,\omega \right)\left[-\omega \frac{d}{dz}\delta a^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)\delta a^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ =-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(1;2;z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\hspace{0.17em}+\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }\right.{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)}\\ \times \left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)\right.{\left.\left.+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(183)

Integrating by parts the left side of Equation (183) over the independent variable z yields the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(1;2;z,\omega \right)\left[\omega \frac{d{v}^{\left(1\right)}\left(z,\omega \right)}{dz}+g_2\left(\alpha \right)v^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{1em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(2;2;z,\omega \right)\left[-\omega \frac{d}{dz}\delta a^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)\delta a^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ ={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }{\left\{\hspace{0.17em}{a}^{(2)}\left(1;2;b,\omega \right)v^{\left(1\right)}\left(b,\omega \right)-\hspace{0.17em}{a}^{(2)}\left(1;2;a,\omega \right)v^{\left(1\right)}\left(a,\omega \right)\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{v}^{\left(1\right)}\left(z,\omega \right)\left[-\omega \frac{d}{dz}{a}^{(2)}\left(1;2;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(1;2;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\\ +{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }{\left\{\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(2;2;a,\omega \right)\delta a^{\left(1\right)}\left(a,\omega \right)-a^{(2)}\left(2;2;b,\omega \right)\delta a^{\left(1\right)}\left(b,\omega \right)\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\delta a^{\left(1\right)}\left(z,\omega \right)\left[\omega \frac{d}{dz}{a}^{(2)}\left(2;2;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;2;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}.\end{array}$$

(184)

The second and the last term on the right side of Equation (184) are now required to represent the indirect-effect term defined in Equation (181). This requirement is achieved by imposing the following relationships for $a^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1$:

$$\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\left\{-\omega \frac{d}{dz}{a}^{(2)}\left(1;2;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(1;2;z,\omega \right)\right\}}_{{\alpha }^0}=-{\left\{a^{\left(1\right)}\left(z,\omega \right)\right\}}_{{\alpha }^0}$$

(185)

$${\left\{\omega \frac{d}{dz}{a}^{(2)}\left(2;2;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;2;z,\omega \right)\right\}}_{{\alpha }^0}=-{\left\{\phi \left(z,\omega \right)\right\}}_{{\alpha }^0}$$

(186)

The definition of the two-component function $A^{(2)}\left(2;2;x\right)\triangleq {\left[a^{(2)}\left(1;2;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\right]}^{†}$ is now completed by requiring that the unknown values of the functions $v^{\left(1\right)}\left(b,\omega \right)$ and $\delta a^{\left(1\right)}\left(a,\omega \right)$ be eliminated from appearing on the right side of Equation (184). This requirement is met by imposing the following conditions to be satisfied by the function $A^{(2)}\left(2;2;x\right)\triangleq {\left[a^{(2)}\left(1;2;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\right]}^{†}$ on the model’s boundaries:

$$\hspace{0.17em}{a}^{(2)}\left(1;2;z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(187)

$$\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=a^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(188)

Altogether, Equations (185)–(188) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS) for determining the second-level adjoint sensitivity function $A^{(2)}\left(2;2;x\right)\triangleq {\left[a^{(2)}\left(1;2;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\right]}^{†}$. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain $A^{(2)}\left(2;2;x\right)\triangleq {\left[a^{(2)}\left(1;2;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\right]}^{†}$. For the paradigm model under consideration, the 2nd-LASS given by Equations (185)–(188) can be solved exactly to obtain the following closed-form expression for the two non-zero components of the 2nd-level adjoint function $A^{(2)}\left(2;2;z,\omega \right)\triangleq {\left[a^{(2)}\left(1;2;z,\omega \right),\hspace{0.17em}\hspace{1em}\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\right]}^{†}$:

$$\hspace{0.17em}{a}^{(2)}\left(1;2;z,\omega \right)=\frac{h\left(\alpha \right)}{{\omega }^2}\exp \left[\frac{g_2\left(\alpha \right)\left(z-b\right)}{\omega }\right]$$

(189)

$$\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)=\hspace{0.17em}\frac{g_1\left(\alpha \right)}{\omega }\left(a-z\right)\exp \left[\frac{g_2\left(\alpha \right)\left(a-z\right)}{\omega }\right]$$

(190)

Collecting the results obtained in Equations (181)–(188) while using the boundary conditions provided in Equations (113) and (164) leads to the following expression for the indirect-effect term:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(2;\phi ;a^{\left(1\right)};f;\delta a^{\left(1\right)};v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}=-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(1;2;z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]}\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega d\omega }{a}^{(2)}\left(1;2;a,\omega \right)\left[\left(\delta g_1\right)+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}\\ \triangleq {\left\{\delta R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f;A^{(2)}\left(2;2;z,\omega \right)\right]\right\}}_{ind}.\end{array}$$

(191)

Note that the dependence of ${\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ on the variational functions $v^{\left(1\right)}\left(z,\omega \right)$ and $\delta a^{\left(1\right)}\left(z,\omega \right)$ has been replaced in Equation (191) by the dependence on the second-level adjoint sensitivity function $A^{(2)}\left(2;2;z,\omega \right)$.

Adding the results obtained in Equations (191) and (182) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f;\delta a^{\left(1\right)};v^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}=-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(1;2;z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]}\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(1;2;a,\omega \right)\left[\left(\delta g_1\right)+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}+\left(\delta a\right)g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\\ \triangleq {\left\{\delta R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f;A^{(2)}\left(2;2;z,\omega \right)\delta f\right]\right\}}_{{\alpha }^0}.\end{array}$$

(192)

Identifying the expressions that multiply the respective components of $\delta f\left(\alpha \right)\triangleq {\left[\delta g_1\left(\alpha \right),\delta g_2\left(\alpha \right);\delta h\left(\alpha \right);\delta a;\delta b\right]}^{†}$ in Equation (192) and using the expressions of the components of the 2nd-level adjoint sensitivity function $A^{(2)}\left(2;2;z,\omega \right)$ yields the following results for the second-order sensitivities that stem from the first-order sensitivity $R^{\left(1\right)}\left[2;\phi ;a^{\left(1\right)};f\right]\triangleq \partial R\left(\phi ;f\right)/\partial g_2\left(\alpha \right)$:

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)\partial g_2\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(1;2;a,\omega \right)=\left(a-b\right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right];$$

(193)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_2\partial g_2\left(\alpha \right)}=-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(1;2;z,\omega \right)\phi \left(z,\omega \right)dz}-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)a^{\left(1\right)}\left(z,\omega \right)}\\ =-g_1\left(\alpha \right)h\left(\alpha \right)\left(a-b\right)\left(b-a\right)E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]=\frac{\left(b-a\right)g_1\left(\alpha \right)h\left(\alpha \right)}{g_2\left(\alpha \right)}\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right];\end{array}$$

(194)

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial g_2\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\delta \left(z-b\right)}=g_1\left(\alpha \right)\left(a-b\right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]$$

(195)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial g_2\left(\alpha \right)}=g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]+g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega }\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(1;2;a,\omega \right)\\ =g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em}+g_1\left(\alpha \right)h\left(\alpha \right)\left(a-b\right)g_2\left(\alpha \right)E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\\ =g_1\left(\alpha \right)h\left(\alpha \right)\left\{E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]-\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]\right\}\end{array}$$

(196)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial g_2\left(\alpha \right)}=-h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right){\delta }^{\prime }\left(z-b\right)}\\ =-g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]-g_1\left(\alpha \right)h\left(\alpha \right)\left(a-b\right)g_2\left(\alpha \right)E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\\ =g_1\left(\alpha \right)h\left(\alpha \right)\left\{\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]-E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\};\end{array}$$

(197)

The validity of the expressions obtained in Equations (193)–(197) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).

6.3. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities $\partial R\left(\phi ;f\right)/\partial h\left(\alpha \right)$

The 1st-order sensitivity $\partial R\left(\phi ;f\right)/\partial h\left(\alpha \right)$ is defined in Equation (121), which is recast into the following “response-like” form:

$$R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right)\right]\triangleq \frac{\partial R\left(\phi ;f\right)}{\partial h\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)\delta \left(z-b\right)dz}}$$

(198)

It is evident that the 1st-order sensitivity $R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial a$ does not depend on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$, but only depends on the original forward function $\phi \left(z,\omega \right)$. The digit “3” in the list of arguments of $R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right)\right]$ indicates that this first-order sensitivity is the “third to be considered” of the five first-order sensitivities of the model response (with respect to the five components of the “feature” function $f$), which were obtained in Equations (119)–(123).

According to the principles of the 2nd-CASAM presented in Section 5, the second-order sensitivities which stem from $\partial R\left(\phi ;f\right)/\partial h\left(\alpha \right)$ are provided by the first-order G-differential of $R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h$. Applying the definition of the G-differential to Equation (198) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right);v^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}\triangleq {\left\{\frac{d}{d\epsilon }{\left[{\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega {\displaystyle \underset{a+\epsilon \delta a}{\overset{b+\epsilon \delta b}{\int }}\left(\phi +\epsilon v^{\left(1\right)}\right)\delta \left(z-b-\epsilon \delta b\right)dz}}\hspace{0.17em}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\hspace{0.17em}\\ \hspace{0.17em}={\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(199)

where the direct-effect term ${\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}$ depends only on variations $\delta f$, while the indirect-effect term ${\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ depends only on the variational function $v^{\left(1\right)}\left(z,\omega \right)$; these terms are defined, respectively, as follows:

$${\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{\left(\delta b\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}}\right\}}_{{\alpha }^0}\hspace{0.17em}$$

(200)

$${\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\triangleq {\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{v}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-b\right)dz}}\right\}}_{{\alpha }^0}$$

(201)

The direct-effect term defined by Equation (200) can be evaluated at this stage by using Equation (107) to obtain:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{\left(\delta b\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ =-\left(\delta b\right)g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right];\end{array},$$

(202)

The indirect-effect term defined in Equation (201) can be evaluated after having determined the variational function $v^{\left(1\right)}\left(z,\omega \right)$, which is the solution of Equations (112) and (113). As before, the need for solving the respective variational equations is avoided by constructing an appropriate 2nd-LASS, by applying the principles of the 2nd-FASAM. This 2nd-LASS is constructed by forming the inner product of Equation (112) with an as yet undefined function $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$, to obtain the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(1;3;z,\omega \right)\left[\omega \frac{d{v}^{\left(1\right)}\left(z,\omega \right)}{dz}+g_2\left(\alpha \right)v^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}=-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)\phi \left(z,\omega \right)}\right\}}_{{\alpha }^0}\hspace{0.17em}.\end{array}$$

(203)

Note that the first argument of $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$ indicates that only the “ first component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$ refers to the index/label “$j_1=3$”, which indicates that this second-level adjoint sensitivity function will correspond to “the third considered” first-order sensitivity (namely: $R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h$).

Integrating the left side of Equation (203) by parts over the independent variable z yields the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)\left[\omega \frac{d{v}^{\left(1\right)}\left(z,\omega \right)}{dz}+g_2\left(\alpha \right)v^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }{\left\{\hspace{0.17em}{a}^{(2)}\left(1;3;b,\omega \right)v^{\left(1\right)}\left(b,\omega \right)-a^{(2)}\left(1;3;a,\omega \right)v^{\left(1\right)}\left(a,\omega \right)\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{v}^{\left(1\right)}\left(z,\omega \right)\left[-\omega \frac{d}{dz}{a}^{(2)}\left(1;3;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(1;3;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}.\end{array}$$

(204)

The last term on the right side of Equation (204) is now required to represent the indirect-effect term defined in Equation (201), which is achieved by imposing the following relationship over the domain $a^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1$:

$${\left\{-\omega \frac{d}{dz}{a}^{(2)}\left(1;3;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(1;3;z,\omega \right)\right\}}_{{\alpha }^0}={\left\{\delta \left(z-b\right)\right\}}_{{\alpha }^0}\hspace{0.17em}.$$

(205)

The definition of the function $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$ is now completed by requiring that the unknown value of the function $v^{\left(1\right)}\left(b,\omega \right)$ be eliminated from appearing on the right side of Equation (204). This requirement is met by imposing the following condition to be satisfied by the function $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$ on the model’s outer boundary:

$$\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(206)

Altogether, Equations (205) and (206) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS), for determining the second-level adjoint sensitivity function $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$. For the paradigm model under consideration, the 1st-LASS given by Equations (205) and (206) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$:

$$\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)=\left[1-H\left(z-b\right)\right]\frac{1}{\omega }\exp \left[\frac{g_2\left(\alpha \right)\left(z-b\right)}{\omega }\right]$$

(207)

Collecting the results obtained in Equations (201)–(206) leads to the following expression for the indirect-effect term:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}=-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;3;a,\omega \right)\left[\delta g_1+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}\\ \triangleq {\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;a^{(2)}\left(1;3;z,\omega \right)\right)}_{{\alpha }^0}\right\}}_{ind}.\end{array}$$

(208)

Note that the dependence of ${\left\{\delta R^{\left(1\right)}{\left(3;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ on the variational function $v^{\left(1\right)}\left(z,\omega \right)$ has been replaced in Equation (208) by the dependence on the second-level adjoint sensitivity function $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$.

Adding the results obtained in Equations (208) and (202) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right);v^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}=\hspace{0.17em}-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;3;a,\omega \right)\left[\delta g_1+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}\\ +{\left\{\left(\delta b\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}}\right\}}_{{\alpha }^0}\hspace{0.17em}\hspace{0.17em}\triangleq {\left\{\delta R^{\left(1\right)}\left[3;\phi ;f;a^{(2)}\left(1;3;z,\omega \right);\delta f\right]\right\}}_{{\alpha }^0}.\end{array}$$

(209)

Note that the dependence of ${\left\{\delta R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right);v^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}$ on the variational function $v^{\left(1\right)}\left(z,\omega \right)$ has been replaced in Equation (209) by the dependence on $\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)$.

Identifying the expressions that multiply the respective components of $\delta f\left(\alpha \right)\triangleq {\left[\delta g_1\left(\alpha \right),\delta g_2\left(\alpha \right);\delta h\left(\alpha \right);\delta a;\delta b\right]}^{†}$ and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (207) yields the following results for the second-order sensitivities that stem from the first-order sensitivity $R^{\left(1\right)}\left[3;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h$:

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)\partial h\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;3;a,\omega \right)=E_2\left[\left(b-a\right)g_2\left(\alpha \right)\right]$$

(210)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\hspace{0.17em}\partial g_2\left(\alpha \right)\partial h\left(\alpha \right)}=-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)\phi \left(z,\omega \right)dz}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=-g_1\left(\alpha \right)\left(b-a\right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right];\end{array}$$

(211)

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial h\left(\alpha \right)}=0$$

(212)

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial h\left(\alpha \right)}=g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;3;a,\omega \right)\hspace{0.17em}=g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right];$$

(213)

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial h\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}}=-g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]$$

(214)

The validity of the expressions obtained in Equations (210)–(214) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).

6.4. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities $\partial R\left(\phi ;f\right)/\partial a$

The 1st-order sensitivity $\partial R\left(\phi ;f\right)/\partial a$ is defined in Equation (122), which is recast into the following “response-like” form:

$$R^{\left(1\right)}\left[4;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \frac{\partial R\left(\phi ;f\right)}{\partial a}=g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}$$

(215)

It is evident that the 1st-order sensitivity $R^{\left(1\right)}\left[4;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial a$ does not depend on the original forward function $\phi \left(z,\omega \right)$, but only depends on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$. The digit “4” in the list of arguments of $R^{\left(1\right)}\left[4;a^{\left(1\right)};f\left(\alpha \right)\right]$ indicates that this first-order sensitivity is the “fourth to be considered” of the five first-order sensitivities of the model response with respect to the five components of the “feature” function $f\left(\alpha \right)$, which were obtained in Equations (119)–(123).

According to the principles of the 2nd-CASAM presented in Section 5, the second-order sensitivities which stem from $\partial R\left(\phi ;f\right)/\partial a$ are provided by the first-order G-differential of $R^{\left(1\right)}\left[4;a^{\left(1\right)};f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial a$. Applying the definition of the G-differential to Equation (215) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[4;a^{\left(1\right)};f\left(\alpha \right);\delta a^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}\\ \triangleq {\left\{\frac{d}{d\epsilon }{\left[\left(g_1+\epsilon \delta g_1\right)\left(g_2+\epsilon \delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega {\displaystyle \underset{a+\epsilon \delta a}{\overset{b+\epsilon \delta b}{\int }}\left(a^{\left(1\right)}+\epsilon \delta a^{\left(1\right)}\right)\delta \left(z-a-\epsilon \delta a\right)dz}}\hspace{0.17em}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\hspace{0.17em}\\ \hspace{0.17em}={\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(216)

where the direct-effect term ${\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}$ depends only on variations $\delta f$, while the indirect-effect term ${\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ depends only on variations $\delta a^{\left(1\right)}$; these terms are defined, respectively, as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{\left(\delta g_1\right)g_2{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}\right\}}_{{\alpha }^0}\\ +{\left\{\left(\delta g_2\right)g_1{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}\right\}}_{{\alpha }^0}+{\left\{\left(\delta a\right)g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d{a}^{\left(1\right)}\left(z,\omega \right)}{dz}\right]}_{z=a}}\right\}}_{{\alpha }^0}\hspace{0.17em},\end{array}$$

(217)

$${\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\triangleq {\left\{g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\delta a^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}\right\}}_{{\alpha }^0}$$

(218)

The direct-effect term defined by Equation (217) can be evaluated at this stage by using Equations (116) and (159) to obtain:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq \left(\delta a\right){\left\{\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)}{\left(b-a\right)}\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}\hspace{0.17em}\\ +{\left\{\left[\left(\delta g_1\right)g_2\left(\alpha \right)+\left(\delta g_2\right)g_1\left(\alpha \right)\right]h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(219)

The indirect-effect term defined in Equation (218) can be evaluated after having determined the variational function $\delta a^{\left(1\right)}\left(z,\omega \right)$, which is the solution of Equations (163) and (164), but the need for solving variational equations is avoided by constructing an appropriate 2nd-LASS, by applying the principles of the 2nd-FASAM. This 2nd-LASS is constructed by forming the inner product of Equation (163) with an as yet undefined function $\hspace{0.17em}{a}^{(2)}\left(2;j_1=4;z,\omega \right)$, to obtain the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(2;4;z,\omega \right)\left[-\omega \frac{d}{dz}\delta a^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)\delta a^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}=\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }\right.{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right)\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)\right.}\\ {\left.\left.\hspace{1em}+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(220)

Note that the first argument of $a^{(2)}\left(2;1;z,\omega \right)$ indicates that only the “second component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in $a^{(2)}\left(2;4;z,\omega \right)$ refers to the index/label “$j_1=4$,”which indicates that this second-level adjoint sensitivity function will correspond to “the fourth considered” first-order sensitivity.

Integrating by parts the left side of Equation (220) over the independent variable z yields the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{a}^{(2)}\left(2;4;z,\omega \right)\left[-\omega \frac{d}{dz}\delta a^{\left(1\right)}\left(z,\omega \right)+g_2\left(\alpha \right)\delta a^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }{\left\{\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(2;4;a,\omega \right)\delta a^{\left(1\right)}\left(a,\omega \right)-a^{(2)}\left(2;1;b,\omega \right)\delta a^{\left(1\right)}\left(b,\omega \right)\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\delta a^{\left(1\right)}\left(z,\omega \right)\left[\omega \frac{d}{dz}{a}^{(2)}\left(2;4;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;4;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}.\end{array}$$

(221)

The last term on the right side of Equation (221) is now required to represent the indirect-effect term defined in Equation (218) by imposing the following relationship:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\delta a^{\left(1\right)}\left(z,\omega \right)\left[\omega \frac{d}{dz}{a}^{(2)}\left(2;1;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;1;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\hspace{0.17em}{\left\{g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}\delta a^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}\right\}}_{{\alpha }^0}.\end{array}$$

(222)

The relation in Equation (222) implies that the following equation holds, in the weak sense, at the nominal parameter values for $a^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1$:

$${\left\{\omega \frac{d}{dz}{a}^{(2)}\left(2;4;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(2;4;z,\omega \right)\right\}}_{{\alpha }^0}={\left\{g_1\left(\alpha \right)g_2\left(\alpha \right)\delta \left(z-a\right)\right\}}_{{\alpha }^0}\hspace{0.17em}.$$

(223)

The definition of the function $a^{(2)}\left(2;4;z,\omega \right)$ is now completed by requiring that the unknown value of the function $\delta a^{\left(1\right)}\left(a,\omega \right)$ be eliminated from appearing on the right side of Equation (221). This requirement is met by imposing the following condition to be satisfied by the function $a^{(2)}\left(2;4;z,\omega \right)$ on the model’s boundary:

$$a^{(2)}\left(2;4;z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=a^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(224)

Altogether, Equations (223) and (224) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS), for determining the second-level adjoint sensitivity function $a^{(2)}\left(2;4;z,\omega \right)$. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain $a^{(2)}\left(2;4;z,\omega \right)$ For the paradigm model under consideration, the 1st-LASS given by Equations (223) and (224) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function $a^{(2)}\left(2;4;z,\omega \right)$:

$$a^{(2)}\left(2;4;z,\omega \right)=H\left(z-a\right)g_1\left(\alpha \right)g_2\left(\alpha \right)\frac{1}{\omega }\exp \left[\frac{g_2\left(\alpha \right)\left(a-z\right)}{\omega }\right]$$

(225)

Collecting the results obtained in Equations (220)–(225) leads to the following expression for the indirect-effect term:

$$\begin{array}{l}\hspace{0.17em}{\left\{\delta R^{\left(1\right)}{\left(4;a^{\left(1\right)};f;\delta a^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}=\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }\right.{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right)}\\ {\left.\left.\times \hspace{0.17em}\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)\right.+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]\right\}}_{{\alpha }^0}\\ ={\left\{\delta R^{\left(1\right)}{\left[4;a^{\left(1\right)};f;a^{(2)}\left(2;4;z,\omega \right)\right]}_{{\alpha }^0}\right\}}_{ind}.\end{array}$$

(226)

Note that the dependence of ${\left\{\delta R^{\left(1\right)}{\left[4;a^{\left(1\right)};f;a^{(2)}\left(2;4;z,\omega \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ on the variational function $\delta a^{\left(1\right)}$ has been replaced in Equation (226) by the dependence on the second-level adjoint sensitivity function $a^{(2)}\left(2;1;z,\omega \right)$.

Adding the results obtained in Equations (226) and (219) (or Equation (217)) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[4;a^{\left(1\right)};f\left(\alpha \right);a^{(2)}\left(2;4;z,\omega \right);\delta f\right]\right\}}_{{\alpha }^0}=\left(\delta a\right){\left\{\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)}{\left(b-a\right)}\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}\\ +{\left\{\left[\left(\delta g_1\right)g_2\left(\alpha \right)+\left(\delta g_2\right)g_1\left(\alpha \right)\right]h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right)\left.\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\left[-\left(\delta g_2\right)a^{\left(1\right)}\left(z,\omega \right)\right.+\left(\delta h\right)\delta \left(z-b\right)-\left(\delta b\right)h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right]}\right\}}_{{\alpha }^0}.\end{array}$$

(227)

Identifying the expressions that multiply the respective components of $\delta f\left(\alpha \right)\triangleq {\left[\delta g_1\left(\alpha \right),\delta g_2\left(\alpha \right);\delta h\left(\alpha \right);\delta a;\delta b\right]}^{†}$ and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (225) yields the following results for the second-order sensitivities that stem from the first-order sensitivity $\partial R\left(\phi ;f\right)/\partial a$:

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\hspace{0.17em}\left(\alpha \right)\partial a}=g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}=g_2\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]$$

(228)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_2\hspace{0.17em}\left(\alpha \right)\partial a}=g_1\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{\left(1\right)}\left(z,\omega \right)\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right)}}\\ =g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]-g_1\left(\alpha \right)h\left(\alpha \right)g_2\left(\alpha \right)\left(b-a\right)E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\\ =g_1\left(\alpha \right)h\left(\alpha \right)\left\{E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]-\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]\right\}.\end{array}$$

(229)

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial a}={\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right)}\hspace{0.17em}\delta \left(z-b\right)=g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]$$

(230)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial a}=g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d{a}^{\left(1\right)}\left(z,\omega \right)}{dz}\right]}_{z=a}}=\hspace{0.17em}{g}_1\left(\alpha \right){\left[g_2\left(\alpha \right)\hspace{0.17em}\right]}^{2}h\left(\alpha \right)\\ \hspace{0.17em}\hspace{0.17em}\times E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em}\hspace{0.17em}=\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)}{\left(b-a\right)}\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right];\end{array}$$

(231)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial a}=-h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right){\delta }^{\prime }\left(z-b\right)}=-g_1\left(\alpha \right){\left[g_2\left(\alpha \right)\right]}^{2}h\left(\alpha \right)\\ \times E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em}\hspace{0.17em}=-\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)}{\left(b-a\right)}\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right].\end{array}$$

(232)

The validity of the expressions obtained in Equations (228)–(232) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).

6.5. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities $\partial R\left(\phi ;f\right)/\partial b$

The 1st-order sensitivity $\partial R\left(\phi ;f\right)/\partial b$ is defined in Equation (123), which is recast into the following “response-like” form:

$$R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right)\right]\triangleq {\left\{\frac{\partial R\left(\phi ;f\right)}{\partial b}\right\}}_{{\alpha }^0}=-{\left\{h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right)}\hspace{0.17em}}{\delta }^{\prime }\left(z-b\right)dz\right\}}_{{\alpha }^0}$$

(233)

As Equation (233) indicates, the 1st-order sensitivity $R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial b$ does not depend on the first-level adjoint sensitivity function $a^{\left(1\right)}\left(z,\omega \right)$, but only depends on the original forward function $\phi \left(z,\omega \right)$. The digit “5” in the list of arguments of $R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right)\right]$ indicates that this first-order sensitivity is the “fifth to be considered” (of the five first-order sensitivities of the model response with respect to the five components of the “feature” function $f$), which were obtained in Equations (119)–(123).

According to the principles of the 2nd-CASAM presented in Section 5, the second-order sensitivities which stem from $\partial R\left(\phi ;f\right)/\partial b$ are provided by the first-order G-differential of $R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial b$. Applying the definition of the G-differential to Equation (233) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right);v^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}\\ \triangleq -{\left\{\frac{d}{d\epsilon }{\left[\left(h+\epsilon \delta h\right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega {\displaystyle \underset{a+\epsilon \delta a}{\overset{b+\epsilon \delta b}{\int }}\left(\phi +\epsilon v^{\left(1\right)}\right){\delta }^{\prime }\left(z-b-\epsilon \delta b\right)dz}}\hspace{0.17em}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\hspace{0.17em}\\ \hspace{0.17em}={\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(234)

where the direct-effect term ${\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}$ depends only on variations $\delta f$, while the indirect-effect term ${\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ depends only on variations $v^{\left(1\right)}\left(z,\omega \right)$; these terms are defined, respectively, as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}\triangleq -{\left\{\left(\delta h\right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right){\delta }^{\prime }\left(z-b\right)dz}}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\left(\delta b\right)h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}\phi \left(z,\omega \right){\delta }^{″}\left(z-b\right)dz}};\end{array}$$

(235)

$${\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\triangleq -{\left\{h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{v}^{\left(1\right)}\left(z,\omega \right){\delta }^{\prime }\left(z-b\right)dz}}\right\}}_{{\alpha }^0}$$

(236)

The direct-effect term defined by Equation (235) can be evaluated at this stage by using Equation (107) to obtain:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;\delta f\right)}_{{\alpha }^0}\right\}}_{dir}={\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega }{\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}\right\}}_{{\alpha }^0}\\ +{\left\{\left(\delta b\right)h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\left[\frac{d^2\phi \left(z,\omega \right)}{d{z}^2}\right]}_{z=b}\right\}}_{{\alpha }^0}=-{\left\{\left(\delta h\right)g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}\\ +{\left\{\left(\delta b\right)g_1\left(\alpha \right)h\left(\alpha \right){\left[g_2\left(\alpha \right)\right]}^2{E}_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}}_{{\alpha }^0}.\end{array}$$

(237)

The indirect-effect term defined in Equation (236) can be evaluated after having determined the variational function $v^{\left(1\right)}\left(z,\omega \right)$, which is the solution of Equations (112) and (113), but the need to solve the respective variational equations is avoided by constructing an appropriate 2nd-LASS, by applying the principles of the 2nd-FASAM. This 2nd-LASS is constructed by forming the inner product of Equation (112) with an as yet undefined function $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$, to obtain the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)\left[\omega \frac{d{v}^{\left(1\right)}\left(z,\omega \right)}{dz}+g_2\left(\alpha \right)v^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}=-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)\phi \left(z,\omega \right)}\right\}}_{{\alpha }^0}\hspace{0.17em}.\end{array}$$

(238)

Note that the first argument of $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$ indicates that only the “first component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$ refers to the index/label “$j_1=5$”, which indicates that this second-level adjoint sensitivity function will correspond to “the third considered” first-order sensitivity (namely: $R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial h$).

Integrating the left side of Equation (238) by parts over the independent variable z yields the following relation:

$$\begin{array}{l}{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)\left[\omega \frac{d{v}^{\left(1\right)}\left(z,\omega \right)}{dz}+g_2\left(\alpha \right)v^{\left(1\right)}\left(z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }{\left\{\hspace{0.17em}{a}^{(2)}\left(1;5;b,\omega \right)v^{\left(1\right)}\left(b,\omega \right)-a^{(2)}\left(1;5;a,\omega \right)v^{\left(1\right)}\left(a,\omega \right)\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}{v}^{\left(1\right)}\left(z,\omega \right)\left[-\omega \frac{d}{dz}{a}^{(2)}\left(1;5;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(1;5;z,\omega \right)\right]dz}\right\}}_{{\alpha }^0}.\end{array}$$

(239)

The last term on the right side of Equation (239) is now required to represent the indirect-effect term defined in Equation (236), which is achieved by imposing the following relationship over the domain $a^0<z<b^0;\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1$:

$${\left\{-\omega \frac{d}{dz}{a}^{(2)}\left(1;5;z,\omega \right)+g_2\left(\alpha \right)a^{(2)}\left(1;5;z,\omega \right)\right\}}_{{\alpha }^0}=-{\left\{h\left(\alpha \right){\delta }^{\prime }\left(z-b\right)\right\}}_{{\alpha }^0}\hspace{0.17em}.$$

(240)

The definition of the function $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$ is now completed by requiring that the unknown value of the function $v^{\left(1\right)}\left(b,\omega \right)$ be eliminated from appearing on the right side of Equation (239). This requirement is fulfilled by imposing the following condition to be satisfied by the function $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$ on the model’s outer boundary:

$$\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)=0,\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}z=b^0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\le \omega \le 1.$$

(241)

Altogether, Equations (240) and (241) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS) for determining the second-level adjoint sensitivity function $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$. For the paradigm model under consideration, the 1st-LASS given by Equations (240) and (241) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$:

$$\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)=\frac{h\left(\alpha \right)}{\omega }\delta \left(z-b\right)+\left[H\left(z-b\right)-1\right]\frac{h\left(\alpha \right)g_2\left(\alpha \right)}{{\omega }^2}\exp \left[\frac{g_2\left(\alpha \right)\left(z-b\right)}{\omega }\right]$$

(242)

Collecting the results obtained in Equations (236)–(241) leads to the following expression for the indirect-effect term:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}=-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;5;a,\omega \right)\left[\left(\delta g_1\right)+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}\\ \triangleq {\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;a^{(2)}\left(1;5;z,\omega \right)\right)}_{{\alpha }^0}\right\}}_{ind}.\end{array}$$

(243)

Note that the dependence of ${\left\{\delta R^{\left(1\right)}{\left(5;\phi ;f;v^{\left(1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ on the variational function $v^{\left(1\right)}\left(z,\omega \right)$ has been replaced in Equation (243) by the dependence on the second-level adjoint sensitivity function $\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)$.

Adding the results obtained in Equations (243) and (237) yields the following relation:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right);v^{\left(1\right)};\delta f\right]\right\}}_{{\alpha }^0}=\hspace{0.17em}-{\left\{\left(\delta g_2\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)\phi \left(z,\omega \right)dz}\right\}}_{{\alpha }^0}\\ +{\left\{{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;5;a,\omega \right)\left[\left(\delta g_1\right)+\left(\delta a\right)\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)}{\omega }\right]\right\}}_{{\alpha }^0}\\ +{\left\{\left(\delta h\right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}}\right\}}_{{\alpha }^0}+\hspace{0.17em}\hspace{0.17em}{\left\{\left(\delta b\right)h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\left[\frac{d^2\phi \left(z,\omega \right)}{d{z}^2}\right]}_{z=b}\right\}}_{{\alpha }^0}\\ \triangleq {\left\{\delta R^{\left(1\right)}\left[5;\phi ;f;a^{(2)}\left(1;3;z,\omega \right);\delta f\right]\right\}}_{{\alpha }^0}.\end{array}$$

(244)

Identifying the expressions that multiply the respective components of $\delta f\left(\alpha \right)\triangleq {\left[\delta g_1\left(\alpha \right),\delta g_2\left(\alpha \right);\delta h\left(\alpha \right);\delta a;\delta b\right]}^{†}$ and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (242) yields the following results for the second-order sensitivities that stem from the first-order sensitivity $R^{\left(1\right)}\left[5;\phi ;f\left(\alpha \right)\right]\triangleq \partial R\left(\phi ;f\right)/\partial b$:

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\hspace{0.17em}\left(\alpha \right)\partial b}=\hspace{0.17em}{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{(2)}\left(1;5;a,\omega \right)\omega \hspace{0.17em}d\omega }\hspace{0.17em}=-g_2\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]$$

(245)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)\partial b}=-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)\phi \left(z,\omega \right)dz}\\ =-g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]+g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)\left(b-a\right)E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\\ =g_1\left(\alpha \right)h\left(\alpha \right)\left\{-E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]+\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]\right\};\end{array}$$

(246)

$$\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial b}={\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega }{\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}=-g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]$$

(247)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial b}=g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}{a}^{(2)}\left(1;5;a,\omega \right)d\omega }=-g_1\left(\alpha \right)h\left(\alpha \right){\left[g_2\left(\alpha \right)\right]}^2\\ \times E_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]=-\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)}{\left(b-a\right)}\exp \left[\left(b-a\right)g_2\left(\alpha \right)\right];\end{array}$$

(248)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial b}=h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\left[\frac{d^2\phi \left(z,\omega \right)}{d{z}^2}\right]}_{z=b}=g_1\left(\alpha \right)h\left(\alpha \right){\left[g_2\left(\alpha \right)\right]}^2{E}_0\left[\left(b-a\right)g_2\left(\alpha \right)\right]\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)}{\left(b-a\right)}\exp \left\{\left(a-b\right)g_2\left(\alpha \right)\right\}\hspace{0.17em}.\end{array}$$

(249)

The validity of the expressions obtained in Equations (245)–(249) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).

6.6. Comparing Computational Efficiencies: The 2nd-FASAM vs. 2nd-CASAM

As has been shown in the foregoing Section 6.1–6.5, each of the five first-order sensitivities of the model response to the components of the feature function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$ has given rise to five second-order sensitivities. The ten mixed second-order sensitivities have been computed twice, each time using two distinct second-level adjoint sensitivity functions, as follows:

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)\partial g_1\left(\alpha \right)}=-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)a^{\left(1\right)}\left(z,\omega \right)}}=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)\partial g_2\left(\alpha \right)}\\ ={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(1;2;a,\omega \right)=\left(a-b\right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em}\hspace{0.17em};\end{array}$$

(250)

$$\begin{array}{l}\hspace{0.17em}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial g_1\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)\delta \left(z-b\right)}=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\left(\alpha \right)\partial h\left(\alpha \right)}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;3;a,\omega \right)=E_2\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em}\hspace{0.17em};\end{array}$$

(251)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial g_1\left(\alpha \right)}=\hspace{0.17em}{\displaystyle \underset{0}{\overset{1}{\int }}\omega \hspace{0.17em}d\omega {\left[\frac{d{a}^{\left(1\right)}\left(z,\omega \right)}{dz}\right]}_{z=a}}=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\hspace{0.17em}\left(\alpha \right)\partial a}\\ =g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}}=g_2\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em};\end{array}$$

(252)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial g_1\left(\alpha \right)}\hspace{0.17em}=-h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;1;z,\omega \right)}\hspace{0.17em}{\delta }^{\prime }\left(z-b\right)=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_1\hspace{0.17em}\left(\alpha \right)\partial b}\\ =\hspace{0.17em}{\displaystyle \underset{0}{\overset{1}{\int }}{a}^{(2)}\left(1;5;a,\omega \right)\omega \hspace{0.17em}d\omega }\hspace{0.17em}=-g_2\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em};\end{array}$$

(253)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial g_2\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right)\delta \left(z-b\right)}=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\hspace{0.17em}\partial g_2\left(\alpha \right)\partial h\left(\alpha \right)}\\ =-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;3;z,\omega \right)\phi \left(z,\omega \right)dz}=g_1\left(\alpha \right)\left(a-b\right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em}\hspace{0.17em};\end{array}$$

(254)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial g_2\left(\alpha \right)}=g_1\left(\alpha \right)h\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]+g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega }\hspace{0.17em}\hspace{0.17em}{a}^{(2)}\left(1;2;a,\omega \right)\\ =\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_2\hspace{0.17em}\left(\alpha \right)\partial a}=g_1\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\displaystyle \underset{a}{\overset{b}{\int }}{a}^{\left(1\right)}\left(z,\omega \right)\delta \left(z-a\right)dz}-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{\left(1\right)}\left(z,\omega \right)\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right)}}\\ =g_1\left(\alpha \right)h\left(\alpha \right)\left\{E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]-\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]\right\}\hspace{0.17em}\hspace{0.17em};\end{array}$$

(255)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial g_2\left(\alpha \right)}=-h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;2;z,\omega \right){\delta }^{\prime }\left(z-b\right)}\\ =\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial g_2\left(\alpha \right)\partial b}=-{\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}\hspace{0.17em}{a}^{(2)}\left(1;5;z,\omega \right)\phi \left(z,\omega \right)dz}\\ =g_1\left(\alpha \right)h\left(\alpha \right)\left\{\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]-E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\right\}\hspace{0.17em}\hspace{0.17em};\end{array}$$

(256)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial h\left(\alpha \right)}=g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega }\hspace{0.17em}{a}^{(2)}\left(1;3;a,\omega \right)\hspace{0.17em}=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial a}\\ ={\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right)}\hspace{0.17em}\delta \left(z-b\right)=g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em}\hspace{0.17em};\end{array}$$

(257)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial h\left(\alpha \right)}={\displaystyle \underset{0}{\overset{1}{\int }}d\omega {\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}}=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial h\left(\alpha \right)\partial b}\\ ={\displaystyle \underset{0}{\overset{1}{\int }}\hspace{0.17em}d\omega }{\left[\frac{d\phi \left(z,\omega \right)}{dz}\right]}_{z=b}=-g_1\left(\alpha \right)g_2\left(\alpha \right)E_1\left[\left(b-a\right)g_2\left(\alpha \right)\right]\hspace{0.17em};\end{array}$$

(258)

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial b\partial a}=-h\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}d\omega }{\displaystyle \underset{a}{\overset{b}{\int }}dz\hspace{0.17em}{a}^{(2)}\left(2;4;z,\omega \right){\delta }^{\prime }\left(z-b\right)}=\frac{{\partial }^{2}R\left(\phi ;f\right)}{\partial a\partial b}\\ =g_1\left(\alpha \right)g_2\left(\alpha \right){\displaystyle \underset{0}{\overset{1}{\int }}{a}^{(2)}\left(1;5;a,\omega \right)d\omega }\hspace{0.17em}\hspace{0.17em}=-\frac{g_1\left(\alpha \right)g_2\left(\alpha \right)h\left(\alpha \right)}{\left(b-a\right)}\exp \left[\left(a-b\right)g_2\left(\alpha \right)\right]\hspace{0.17em}.\end{array}$$

(259)

As the relations shown in Equations (250)–(259) indicate, the 2nd-FASAM provides an intrinsic verification, based on the symmetries inherent to the mixed second-order sensitivities, of having correctly solved the original forward system, the 1st-LASS, and the five 2nd-LASS for the five second-level adjoint sensitivity functions, all of which are involved in the expressions of the mixed second-order sensitivities.

The determination of the second-order response sensitivities to the primary model parameters $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}$, where $\hspace{0.17em}TP=2MA+2MB+2MD+MQ+2$, defined in Equation (72), is accomplished by using the chain rule provided in Equation (154), together with the expressions of the five first-order response sensitivities with respect to the components of the feature function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$, as provided in Section 4.2, Equations (119)–(123), and the second-order response sensitivities with respect to the components of the feature function provided in Section 6.1, Section 6.2, Section 6.3, Section 6.4 and Section 6.5.

Alternatively, it would have been possible to apply the 2nd-CASAM [7,8] to determine the second-order sensitivities (of the response) directly, with respect to the primary parameters $\alpha \triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}$, by using the $\hspace{0.17em}TP=2MA+2MB+2MD+MQ+2$ first-order sensitivities with respect to the primary parameters. Evidently, following this procedure would have required $\hspace{0.17em}TP=2MA+2MB+2MD+MQ+2$ large-scale computations for solving the TP 2nd-LASSystems, each of which would have corresponded to one of the $TP$ first-order sensitivities. It is evident that two of these computations would stem from the first-order sensitivities to the boundary parameters $a$ and $b$, which are also responsible for two of the five 2nd-LASS computations involved in the 2nd-FASAM; recall that the “feature” function $f\left(\alpha \right)\triangleq {\left[g_1\left(\alpha \right),g_2\left(\alpha \right);h\left(\alpha \right);a;b\right]}^{†}$ comprises just three bona fide “features”, namely $g_1\left(\alpha \right)$, $g_2\left(\alpha \right)$ and $h\left(\alpha \right)$.

Even for the simple paradigm shielding model considered in this work, it is evident that the 2nd-FASAM is computationally significantly more advantageous to use rather than the 2nd-CASAM: the 2nd-FASAM requires five large-scale computations (plus insignificant additional analytic computations for applying the chain rule) to obtaining all of the second-order sensitivities with respect to the primary model parameters, while the 2nd-CASAM would require $\hspace{0.17em}TP>70$ large-scale computations to obtain these second-order sensitivities. By comparison, even the simplest-minded four-point finite-difference scheme would require $\hspace{0.17em}4\times TP\times \left(TP+1\right)/2=O\left({10}^4\right)$ large-scale computations, and would at best provide approximate values (whin second-order errors of the finite-difference step-size) for the respective second-order sensitivities.

7. Conclusions

This work has introduced the newly developed “First-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (1st-FASAM) and the “Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (2nd-FASAM). These methodologies enable the most efficient computation of exactly obtained expressions of the first-order (1st-FASAM) and, respectively, second-order sensitivities (2nd-FASAM) of a model response with respect to a function (“feature”) of primary model parameters. For the computation of such first-order sensitivities, the 1st-FASAM requires a single large-scale computation, which is needed for solving the “first-level adjoint sensitivity system (1st-LASS). This is similar to the computational requirement needed by the extant 1st-CASAM [7,8] for obtaining the sensitivities to the primary model parameters. For computing the second-order sensitivities, however, the 2nd-FASAM requires only as many large-scale (adjoint) computations as there are component “features.” In contradistinction, the 2nd-CASAM requires as many large-scale (adjoint) computations as there are primary parameters. Since the number of “features” in a computational model is invariably smaller than the number of primary model parameters, the 2nd-FASAM is always more advantageous to use, computationally, than the 2nd-CASAM. All of these considerations were demonstrated in this work by using a simple particle transport/shielding model which admits closed-form analytic expressions for all of the first- and second-order sensitivities.

In all cases, however, both the 2nd-FASAM and/or the 2nd-CASAM are vastly more efficient to use than conventional finite-difference or statistical methods, which are not only exorbitantly expensive computationally but cannot produce exact expressions for any sensitivity because of their very foundation. Ongoing work aims at extending/generalizing the 2nd-FASAM to enable the computation, with unparalleled efficiency, of exactly-obtained expressions of model response sensitivities of an arbitrarily high-order with respect to functions/features of primary model parameters. The gains in computational efficiency are expected to increase exponentially with the increase in the order of the respective sensitivities. It is envisaged that these innovative aspects and computational gains will be illustrated by using graphs and tables in conjunction with both analytical and numerical examples.