The n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems (n^th-CASAM-L): I. Mathematical Framework

Abstract

This work presents the mathematical framework of the n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems (abbreviated as “n^th-CASAM-L”), which is conceived for obtaining the exact expressions of arbitrarily-high-order (n^th-order) sensitivities of a generic system response with respect to all of the parameters (including boundary and initial conditions) underlying the respective forward/adjoint systems. Since many of the most important responses for linear systems involve the solutions of both the forward and the adjoint linear models that correspond to the respective physical system, the sensitivity analysis of such responses makes it necessary to treat linear systems in their own right, rather than treating them as particular cases of nonlinear systems. This is in contradistinction to responses for nonlinear systems, which can depend only on the forward functions, since nonlinear operators do not admit bona-fide adjoint operators (only a linearized form of a nonlinear operator admits an adjoint operator). The n^th-CASAM-L determines the exact expression of arbitrarily-high order sensitivities of responses to the parameters underlying both the forward and adjoint models of a nonlinear system, thus enable the most efficient and accurate computation of such sensitivities. The mathematical framework underlying the n^th-CASAM is developed in linearly increasing higher-dimensional Hilbert spaces, as opposed to the exponentially increasing “parameter-dimensional” spaces in which response sensitivities are computed by other methods, thus providing the basis for overcoming the “curse of dimensionality” in sensitivity analysis and all other fields (uncertainty quantification, predictive modeling, etc.) which need such sensitivities. In particular, for a scalar-valued valued response associated with a nonlinear model comprising TP parameters, the 1^st-⁻CASAM-L requires 1 additional large-scale adjoint computation (as opposed to TP large-scale computations, as required by other methods) for computing exactly all of the 1^st-⁻order response sensitivities. All of the (mixed) 2^nd-order sensitivities are computed exactly by the 2^nd-CASAM-L in at most TP computations, as opposed to TP(TP + 1)/2 computations required by all other methods, and so on. For every lower-order sensitivity of interest, the n^th-CASAM-L computes the “TP next-higher-order” sensitivities in one adjoint computation performed in a linearly increasing higher-dimensional Hilbert space. Very importantly, the n^th-CASAM-L computes the higher-level adjoint functions using the same forward and adjoint solvers (i.e., computer codes) as used for solving the original forward and adjoint systems, thus requiring relatively minor additional software development for computing the various-order sensitivities.

1. Introduction

The functional derivatives of results (customarily called “responses”) produced by computational models of physical systems with respect to the model’s parameters are customarily called the “response sensitivities.” The first-order sensitivities have been used for a variety of purposes, including: (i) understanding the model by ranking the importance of the various parameters; (ii) designing and optimizing the system; (iii) performing “reduced-order modeling” by eliminating unimportant parameters; (iv) quantifying the uncertainties induced in a model response due to model parameter uncertainties; (v) performing data assimilation and model calibration; (vi) performing “model validation” by comparing computations to experiments to address the question “does the model represent reality?” (vii) prioritizing improvements in the model; (viii) performing “forward and inverse” predictive modeling” to obtain best-estimate predicted results with reduced predicted uncertainties.

As is well known, non-linear operators do not admit adjoint operators; only linear operators admit corresponding adjoint operators. For this reason, many of the most important responses for linear systems involve the solutions of both the forward and the adjoint linear models that correspond to the respective physical system. Included among the widest used system responses that involve both the forward and adjoint functions are the various forms of Lagrangian functionals, the Raleigh quotient for computing eigenvalues and/or separation constants when solving partial differential equations, the Schwinger functional for first-order “normalization-free” solutions [1,2,3,4,5]. These functionals play a fundamental role in optimization and control procedures, derivation of numerical methods for solving equations (differential, integral, integro-differential), etc. The analysis of responses that simultaneously involve both forward and adjoint functions makes it necessary to treat linear systems in their own right, rather than treating them as particular cases of nonlinear systems. This is in contradistinction to responses for nonlinear systems, which can depend only on the forward functions, since nonlinear operators do not admit bona-fide adjoint operators—only a linearized form of a nonlinear operator admits an adjoint operator.

As is well known, even the approximate determination of the first-order sensitivities $\partial R/\partial {\alpha }_i, i=1,\dots ,TP$ of a model response $R$ to $TP$ parameters ${\alpha }_i$ using conventional finite-difference methods would require at least $TP$ large-scale computations with altered parameter values. The computation of the distinct second-order response sensitivities of a would require $TP\left(TP+1\right)/2$ large-scale computations, which rapidly becomes unfeasible for large-scale models comprising many parameters, even using supercomputers. The computation of higher-order sensitivities by conventional methods is limited in practice by the so-called “curse of dimensionality” [6] since the number of large-scale computations needed by conventional, including statistical, methods for computing higher-order response sensitivities increases exponentially with the order of the response sensitivities.

This work presents the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward and Adjoint Linear Systems,” which will be abbreviated as “n^th-CASAM-L.” The n^th-CASAM-L enables the efficient computation of the exact expressions of the 1^st-, 2^nd-, 3^rd- and 4^th-order sensitivities of a generic system response which depends on both the forward and adjoint state functions with respect to all of the parameters underlying the respective systems. The mathematical framework of the n^th-CASAM-L is constructed upon the fundamental principles introduced by Cacuci [7,8,9,10,11] for the first- and second-order adjoint sensitivity analysis methodology. The qualifier “comprehensive” is used because the original methodology [7,8,9,10,11] was extended to provide exact expressions for the sensitivities of a system response not only to the system’s internal parameters but also to the system’s uncertain boundaries and internal interfaces in phase-space [12,13]. The principles underlying the 2^nd-CASAM-L were applied to an OECD/NEA reactor physics benchmark (a reflected plutonium sphere), for which the response of interest was the experimentally-measured [14] neutron leakage through the sphere’s outer surface. The nuclear data used in the computational modeling of the neutron transport processes through this spherical benchmark included 21,976 imprecisely known model parameters, as follows: 180 group-averaged total microscopic cross sections, 21,600 group-averaged scattering microscopic cross sections, 120 parameters describing the fission process, 60 parameters describing the fission spectrum, 10 parameters describing the system’s sources, and 6 isotopic number densities. The results obtained for the 1^st- and 2^nd-order sensitivities of this benchmark’s leakage response to all of these 21,976 uncertain model parameters were presented in [15,16,17,18,19,20]. Notably, the 2^nd-order sensitivities caused the expected value of the leakage response to be significantly larger than the computed value of the leakage response. The importance of the 2^nd-order sensitivities increased as the relative standard deviations for the cross sections increase. For fully correlated cross sections, for example, neglecting the 2^nd-order sensitivities would cause an error as large as 2000% in the expected value of the leakage response, and up to 6000% in the variance of the leakage response.

Subsequently, Cacuci [21] has developed the 3^rd-order CASAM-L, which was applied to compute [22,23,24] the largest 3^rd-order sensitivities of the OECD/NEA benchmark, which turned out to be among the (180)³ = 5,832,000 third-order mixed sensitivities (of which 988,260 are distinct) of this benchmark’s leakage response with respect to the benchmark’s 180 microscopic total cross sections. The results obtained in [22,23,24] indicated that the absolute value of the largest 3^rd-order relative sensitivity is ca. 437 times larger than the largest 2^nd-order sensitivity and is ca. 20,000 times larger than the largest 1^st-order sensitivity. These results have motivated the recent development by Cacuci [25] of the 4^th-order CASAM-L, which was applied [26,27] to compute the 45,212,895 distinct fourth-order sensitivities of this benchmark’s leakage response with respect to the benchmark’s 180 microscopic total cross sections.

This work is structured as follows: Section 2 presents the general mathematical formulation of the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (n^th-CASAM-L). This innovative methodology has no comparable counterpart in the literature and is applicable to any linear system. The general mathematical formulation of the n^th-CASAM-L will be proven to be correct by using “proof by induction,” which comprises the usual steps, namely: (i) establish the pattern underlying the n^th-CASAM-L for n = 1; (ii) assume that the pattern is valid for an arbitrarily high-order, n; (iii) prove that the pattern is valid for n + 1. The starting point for this proof by induction is the 4^th-order CASAM-L developed by Cacuci [25], which is summarized, for convenient referencing, in Section 4 and Section 5.

Section 3 presents the framework of the “5^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology Response for Coupled Forward/Adjoint Linear Systems” (5^th-CASAM-L). The 5^th-CASAM-L will be developed from the 4^th-CASAM-L, and its mathematical formalism will also be shown to coincide with the particular formulas obtained by setting n = 5 in the general n^th-CASAM-L framework. The 5^th-CASAM-L is also innovative and has no counterpart in the literature. Finally, Section 4 offers conclusions regarding the significance of this work’s novel results in the quest to overcome the curse of dimensionality in sensitivity analysis, uncertainty quantification and predictive modeling. The application of the n^th-CASAM-L is illustrated in an accompanying work [28] by considering a paradigm model which describes the transmission of particles produced by a distributed source through a shield which surrounds the source.

2. The n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems (n^th-CASAM-L)

The general framework of the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems (n^th-CASAM-L)” will be established by using a “proof by induction.” This proof will comprise the usual steps, as follows:

• Establish the pattern underlying the n^th-CASAM-L for n = 1;
• Assume that the pattern is valid for an arbitrarily high-order, n;
• Prove that the pattern is valid for n + 1.

2.1. The Pattern Underlying the n^th-CASAM-L for n = 1, 2, 3, 4

The mathematical framework of the 4^th-CASAM-L, which also comprises the results for the 1^st-, 2^nd-, and 3^rd-CASAM-L, has been conceived by Cacuci [25]. For convenient referencing, the pattern underlying the mathematical framework of the 4^th-CASAM-L is succinctly reviewed in Section 4 and Section 5. The main features of the structures of the mathematical frameworks for n = 1, 2, 3, 4, are summarized in

Table 1. First-Order Sensitivities (n = 1).

Original System; Hilbert Space; Inner Product; Response	$F^{\left(1\right)}\left(\alpha ;x\right)u^{\left(1\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(1\right)}\left(x;\alpha \right)$; $\hspace{0.17em}{b}_F^{\left(1\right)}\left(u^{\left(1\right)};\hspace{0.17em}\alpha \right)=0$; $u^{\left(1\right)}\left(x\right)\triangleq {\left[\phi \left(x\right),\psi \left(x\right)\right]}^{†}$ ${\mathsf{H}}_0$; ${〈\phi \left(x\right),\psi \left(x\right)〉}_0\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\phi \left(x\right)·\psi \left(x\right)\hspace{0.17em}dx}}$; $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$
1st-LVSS	${\left[V^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}\delta u^{\left(1\right)}\left(j_1;x\right)=q_V^{\left(1\right)}\left(j_1;u^{\left(1\right)};\alpha ;\delta \alpha \right)$;$\delta u^{\left(1\right)}\left(x\right)\triangleq {\left[\delta \phi \left(x\right),\delta \psi \left(x\right)\right]}^{†}$; ${\left[V^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}\triangleq F\left(\alpha ;x\right)$; $b_V^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta u^{\left(1\right)}\hspace{0.17em};\delta \alpha \right)=\hspace{0.17em}\delta b_F\left(u^{\left(1\right)};\hspace{0.17em}\alpha ;\delta u^{\left(1\right)}\hspace{0.17em};\delta \alpha \right)=0$
1st-Level Hilbert Space	${\mathsf{H}}_1$; ${〈{\eta }^{\left(1\right)}\left(x\right),{\xi }^{\left(1\right)}\left(x\right)〉}_1\triangleq {\displaystyle \sum _{i=1}^2{〈{\eta }_i^{\left(1\right)}\left(x\right),{\xi }_i^{\left(1\right)}\left(x\right)〉}_0}$
1st-LASS	${\left[A^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}{a}^{\left(1\right)}\left(x\right)\hspace{0.17em}=s_A^{\left(1\right)}\left[u^{\left(1\right)}\left(x\right);\alpha \right]$; ${\left[A^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}\triangleq {\left[V^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}^{*}$; $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$; $b_A^{\left(1\right)}\left(u^{\left(1\right)};a^{\left(1\right)};\alpha \right)=0$
1st-Order Sensitivities:	$R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]\triangleq \partial R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}$; $j_1=1,\dots ,TP$

,

Table 2. Second-Order Sensitivities (n = 2).

2nd-Level Forward System	$\mathrm{Original} \mathrm{Forward} \mathrm{System}+1^{\mathrm{st}}-\mathrm{LASS}: F^{\left(2\right)}\left(\alpha \right)u^{\left(2\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)$ $u^{\left(2\right)}\left(x\right)\triangleq {\left[u^{\left(1\right)}\left(x\right),a^{\left(1\right)}\left(x\right)\right]}^{†}$;$F^{\left(2\right)}\left(\alpha \right)\triangleq diag\left(F^{\left(1\right)},A^{\left(1\right)}\right)$; $\hspace{0.17em}{b}_F^{\left(2\right)}\left(u^{\left(2\right)};\hspace{0.17em}\alpha \right)\triangleq {\left(\hspace{0.17em}{b}_F^{\left(1\right)},b_A^{\left(1\right)}\right)}^{†}={\left[0,0\right]}^{†}$;
2nd-LVSS	${\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}\delta u^{\left(2\right)}\left(x\right)=q_V^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)$ $\delta u^{\left(2\right)}\left(x\right)\triangleq {\left[\delta u^{\left(1\right)}\left(x\right),\delta a^{\left(1\right)}\left(x\right)\right]}^{†}$; ${\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}\triangleq {\left(\begin{array}{cc}{V}^{\left(1\right)}& {\left[0\right]}_{2\times 2}\\ V_{21}^{\left(2\right)}& V_{22}^{\left(2\right)}\end{array}\right)}_{2^2\times 2^2}$ $b_V^{\left(2\right)}\left[u^{\left(2\right)};\alpha ;\delta u^{\left(2\right)}\left(x\right);\delta \alpha \right]\triangleq \left(\begin{array}{c}\hspace{0.17em}{b}_V^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta u^{\left(1\right)};\delta \alpha \right)\\ \delta b_A^{\left(1\right)}\left(u^{\left(2\right)};\alpha ;\delta u^{\left(2\right)};\delta \alpha \right)\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right)$
2nd-Level Hilbert space	${\mathsf{H}}_2$$; {〈{\eta }^{\left(2\right)}\left(x\right),{\xi }^{\left(2\right)}\left(x\right)〉}_2={\displaystyle \sum _{i=1}^4{〈{\eta }_i^{\left(2\right)}\left(x\right),{\xi }_i^{\left(2\right)}\left(x\right)〉}_0}$
2nd-LASS	${\left[A^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}{a}^{\left(2\right)}\left(j_1;x\right)\hspace{0.17em}=s_A^{\left(2\right)}\left(j_1;u^{\left(2\right)};\alpha \right)$$; j_1=1,\dots ,TP$ $a^{\left(2\right)}\left(j_1;x\right)\triangleq {\left[a_1^{\left(2\right)}\left(j_1;x\right),\dots ,a_k^{\left(2\right)}\left(j_1;x\right),\dots ,a_4^{\left(2\right)}\left(j_1;x\right)\right]}^{†};\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^2$ ${\left[A^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}\triangleq {\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}^{*}$$; b_A^{\left(2\right)}\left[u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]=0$
2nd-Order Sensitivities	$R^{\left(2\right)}\left[j_2;j_1;u^{\left(2\right)};a^{\left(2\right)};\alpha \right]\triangleq {\partial }^{2}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}$$;j_2=1,\dots ,j_1$

,

Table 3. Third-Order Sensitivities (n = 3).

3rd-Level Forward System	$2^{\mathrm{nd}}-\mathrm{Level} \mathrm{Forward} \mathrm{System}+2^{\mathrm{nd}}-\mathrm{LASS}; F^{\left(3\right)}{u}^{\left(3\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(3\right)}$; $F^{\left(3\right)}\triangleq diag\left(F^{\left(2\right)},A^{\left(2\right)}\right)$$u^{\left(3\right)}\left(j_1;x\right)={\left[u^{\left(2\right)}\left(x\right),\hspace{0.17em}{a}^{\left(2\right)}\left(j_1;x\right)\right]}^{†}$; $\hspace{0.17em}{b}_F^{\left(3\right)}\left(u^{\left(3\right)};\hspace{0.17em}\alpha \right)\triangleq {\left(\hspace{0.17em}{b}_F^{\left(2\right)},b_A^{\left(2\right)}\right)}^{†}={\left[0,0\right]}^{†}$
3rd-LVSS	${\left[V^{\left(3\right)}\left(j_1\right)\hspace{0.17em}\right]}_{2^3\times 2^3}\delta u^{\left(3\right)}\left(j_1;x\right)\hspace{0.17em}=q_V^{\left(3\right)}\left[j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]$ $\delta u^{\left(3\right)}\left(j_1;x\right)={\left[\delta u^{\left(2\right)}\left(x\right),\delta a^{\left(2\right)}\left(j_1;x\right)\right]}^{†}$; ${\left[V^{\left(3\right)}\right]}_{2^3\times 2^3}\hspace{0.17em}\triangleq \left(\begin{array}{cc}{V}^{\left(2\right)}& {\left[0\right]}_{\left(4\times 4\right)}\\ V_{21}^{\left(3\right)}& V_{22}^{\left(3\right)}\end{array}\right)$ $b_V^{\left(3\right)}\hspace{0.17em}\triangleq {\left[b_V^{\left(2\right)},\delta b_A^{\left(2\right)}\right]}^{†}\hspace{0.17em}={\left[0,0\right]}^{†}$
3rd-Level Hilbert Space	${\mathsf{H}}_3$$; {〈{\eta }^{\left(3\right)}\left(x\right),{\xi }^{\left(3\right)}\left(x\right)〉}_3\triangleq {\displaystyle \sum _{i=1}^{2^3}{〈{\eta }_i^{\left(3\right)}\left(x\right),{\xi }_i^{\left(3\right)}\left(x\right)〉}_0}$
3rd-LASS	$\hspace{0.17em}{\left[A^{\left(3\right)}\left(j_1;u^{\left(2\right)}\right)\right]}_{2^3\times 2^3}{a}^{\left(3\right)}\left(j_2,j_1;x\right)=s_A^{\left(3\right)}\left(j_2,j_1;u^{\left(3\right)};\alpha \right)$ ${\left[A^{\left(3\right)}\left(j_1;u^{\left(2\right)}\right)\right]}_{2^3\times 2^3}\triangleq {\left[V^{\left(3\right)}\left(j_1;u^{\left(2\right)}\right)\right]}_{2^3\times 2^3}^{*}$. $b_A^{\left(3\right)}\left[a^{\left(3\right)}\left(j_2;j_1;x\right);u^{\left(3\right)}\left(j_1;x\right);\alpha \right]=0$ $\begin{array}{l}{a}^{\left(3\right)}\left(j_2,j_1;x\right)\triangleq {\left[a_1^{\left(3\right)}\left(j_2,j_1;x\right),\dots ,a_8^{\left(3\right)}\left(j_2,j_1;x\right)\right]}^{†}\triangleq {\left[\dots ,a_k^{\left(3\right)}\left(j_2,j_1;x\right),\dots \right]}^{†}\\ k=1,\dots ,2^3;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}{j}_2=1,\dots ,j_1;\end{array}$
3rd-Order Sensitivities	$\begin{array}{l}{R}^{\left(3\right)}\left[j_3,j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);a^{\left(3\right)}\left(j_2,j_1;x\right);\alpha \right]\\ \triangleq {\partial }^{3}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}\partial {\alpha }_{j_3};\hspace{0.17em}{j}_3=1,\dots ,j_2\end{array}$

and

Table 4. Fourth-Order Sensitivities (n = 4).

4th-Level Forward System	$3^{\mathrm{rd}}-\mathrm{Level} \mathrm{Forward} \mathrm{System}+3^{\mathrm{rd}}-\mathrm{LASS}: F^{\left(4\right)}{u}^{\left(4\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(4\right)}$; $u^{\left(4\right)}\left(\hspace{0.17em}{j}_2,j_1;x\right)={\left[u^{\left(3\right)}\left(j_1;x\right),\hspace{0.17em}{a}^{\left(3\right)}\left(j_2,j_1;x\right)\right]}^{†}$;$F^{\left(4\right)}\triangleq diag\left(F^{\left(3\right)},A^{\left(3\right)}\right)$ $\hspace{0.17em}{b}_F^{\left(4\right)}\left(u^{\left(4\right)};\hspace{0.17em}\alpha \right)\triangleq {\left(\hspace{0.17em}{b}_F^{\left(3\right)},b_A^{\left(3\right)}\right)}^{†}={\left[0,0\right]}^{†}$
4th-LVSS State Function	$\delta u^{\left(4\right)}\left(j_2,j_1;x\right)={\left[\delta u^{\left(3\right)}\left(j_1;x\right),\delta a^{\left(3\right)}\left(j_2,j_1;x\right)\right]}^{†}$
4th-LVSS	${\left[V^{\left(4\right)}\right]}_{2^4\times 2^4}\delta u^{\left(4\right)}\left(j_2,j_1;x\right)=q_V^{\left(4\right)}\left[j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha ;\delta \alpha \right]$ $b_V^{\left(4\right)}\hspace{0.17em}\triangleq {\left[b_V^{\left(3\right)},\delta b_A^{\left(3\right)}\right]}^{†}\hspace{0.17em}={\left[0,0\right]}^{†}$; ${\left[V^{\left(4\right)}\right]}_{2^4\times 2^4}\triangleq {\left[\begin{array}{cc}{V}^{\left(3\right)}& {\left[0\right]}_{2^3\times 2^3}\\ V_{21}^{\left(4\right)}& V_{22}^{\left(4\right)}\end{array}\right]}_{2^4\times 2^4}$
4th-Level Hilbert Space;	${\mathsf{H}}_4$; ${〈{\eta }^{\left(4\right)}\left(x\right),{\xi }^{\left(4\right)}\left(x\right)〉}_4\triangleq {\displaystyle \sum _{i=1}^{2^4}{〈{\eta }_i^{\left(4\right)}\left(x\right),{\xi }_i^{\left(4\right)}\left(x\right)〉}_0}.$
4th-LASS State Function	$\begin{array}{l}{a}^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\triangleq {\left[a_1^{\left(4\right)}\left(j_3,j_2,j_1;x\right),\dots ,a_{16}^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\right]}^{†}\\ \triangleq {\left[\dots ,a_k^{\left(4\right)}\left(j_3,j_2,j_1;x\right),\dots \right]}^{†};\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^4;\end{array}$
4th-LASS	${\left[A^{\left(4\right)}\hspace{0.17em}\right]}_{2^4\times 2^4}{a}^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\hspace{0.17em}=s_A^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\hspace{0.17em}$; ${\left[A^{\left(4\right)}\right]}_{2^4\times 2^4}\triangleq {\left[V^{\left(4\right)}\right]}_{2^4\times 2^4}^{*}$;$b_A^{\left(4\right)}\left[a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]=0$
4th-Order Sensitivities	$\begin{array}{l}{R}^{\left(4\right)}\left[j_4,j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]\\ \triangleq {\partial }^{4}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\partial {\alpha }_{j_3}\partial {\alpha }_{j_3}\partial {\alpha }_{j_4};\hspace{0.17em}\hspace{0.17em}{j}_4=1,\dots ,j_3\hspace{0.17em}.\end{array}$

. This information will be used in the next Subsection to surmise the general mathematical framework underlying the n^th-CASAM-L, for any order n, to enable the computations of response sensitivities of any order, n.

2.2. The Pattern Underlying the n^th-CASAM-L for Arbitrarily High-Order n

Based on the pattern displayed in Table 1, Table 2, Table 3 and Table 4, the pattern displayed in

Table 5. n^th-Order Sensitivities.

nth-Level Forward System	$(\mathrm{n}-1{)}^{\mathrm{th}}-\mathrm{Level} \mathrm{Forward} \mathrm{System}+(\mathrm{n}-1{)}^{\mathrm{th}}-\mathrm{LASS}: F^{\left(n\right)}{u}^{\left(n\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(n\right)}$; $F^{\left(n\right)}\triangleq diag\left(F^{\left(n-1\right)},A^{\left(n-1\right)}\right)$;$\hspace{0.17em}{b}_F^{\left(n\right)}\left(u^{\left(n\right)};\hspace{0.17em}\alpha \right)\triangleq {\left(\hspace{0.17em}{b}_F^{\left(n-1\right)},b_A^{\left(n-1\right)}\right)}^{†}={\left[0,0\right]}^{†}$ $u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right)={\left[u^{\left(n-1\right)}\left(j_{n-3},\dots ,j_1;x\right),\hspace{0.17em}{a}^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1;x\right)\right]}^{†}$
nth-LVSS	${\left[V^{\left(n\right)}\right]}_{2^n\times 2^n}\delta u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right)=q_V^{\left(n\right)}\left[j_{n-2},\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\alpha ;\delta \alpha \right]$ $\delta u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right)={\left[\delta u^{\left(n-1\right)}\left(j_{n-3},\dots ,j_1;x\right),\delta a^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1;x\right)\right]}^{†}$ $b_V^{\left(n\right)}\hspace{0.17em}\triangleq {\left[b_V^{\left(n-1\right)},\delta b_A^{\left(n-1\right)}\right]}^{†}\hspace{0.17em}={\left[0,0\right]}^{†}$; ${\left[V^{\left(n\right)}\right]}_{2^n\times 2^n}\triangleq {\left(\begin{array}{cc}{V}^{\left(n-1\right)}& {\left[0\right]}_{\left(2^{n-1}\times 2^{n-1}\right)}\\ V_{21}^{\left(n\right)}& V_{22}^{\left(n\right)}\end{array}\right)}_{2^n\times 2^n}$
nth-Level Hilbert Space; Inner Product	${{H}}_n$; ${〈{\eta }^{\left(n\right)}\left(x\right),{\xi }^{\left(n\right)}\left(x\right)〉}_n\triangleq {\displaystyle \sum _{i=1}^{2^n}{〈{\eta }_i^{\left(n\right)}\left(x\right),{\xi }_i^{\left(n\right)}\left(x\right)〉}_0}.$
nth-LASS State Function	$a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right)\triangleq \left[\dots ,a_k^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right),\dots \right];\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^n$
nth-LASS	${\left[A^{\left(n\right)}\hspace{0.17em}\right]}_{2^n\times 2^n}\hspace{0.17em}{a}^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right)\hspace{0.17em}=s_A^{\left(n\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\alpha \right]\hspace{0.17em}$; ${\left[A^{\left(n\right)}\hspace{0.17em}\right]}_{2^n\times 2^n}\hspace{0.17em}\triangleq {\left[V^{\left(n\right)}\right]}_{2^n\times 2^n}^{*}$;$b_A^{\left(n\right)}\left[a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\alpha \right]=0$
No. of Distinct Sensitivities	$TP\left(TP+1\right)\left(TP+2\right)\dots \left(TP+n-1\right)/n!$$\begin{array}{l}{R}^{\left(n\right)}\left[j_n,\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha \right]\\ \triangleq {\partial }^{n}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\dots \partial {\alpha }_{j_n};j_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}{j}_2=1,\dots ,j_1;.....j_n=1,\dots ,j_{n-1}\end{array}$

, below, is expected to be valid for any integer n.

2.3. Proving That the Framewok for the n^th-CASAM-L also Holds for the (n + 1)^th-CASAM-L

The pattern presented in Table 5 implies that the generic n^th-order sensitivity of the response $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ to model parameters has the following form, for $j_1=1,\dots ,TP$;$j_n=1,\dots ,j_{n-1}$;…$j_2=1,\dots ,j_1$:

$$\begin{array}{l}{\partial }^{n}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\dots \partial {\alpha }_{j_n}\triangleq R^{\left(n\right)}\left[j_n,\dots ,j_1;u^{\left(n\right)}\left(j_n,\dots ,j_1;x\right);a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha \right]\\ \triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(n\right)}\left[j_n,\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha \right]}}\end{array}$$

(1)

where the function $u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right)$ is the solution of the following n^th-Level Forward System (n^th-LFS):

$$F^{\left(n\right)}{u}^{\left(n\right)}\left(x\right)= q_F^{\left(n\right)}; b_F^{\left(n\right)}\left(u^{\left(n\right)}; \alpha \right)=0\hspace{1em}, b_F^{\left(n\right)}\left(u^{\left(n\right)}; \alpha \right)=0$$

(2)

and where the n^th-level adjoint function $a^{\left(n\right)}\left(j_{n-1};\dots ;j_1;x\right)$ is the solution of the following n^th-Level Adjoint Sensitivity System (n^th-LASS):

$${\left[A^{\left(n\right)}\right]}_{2^n\times 2^n} a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right) =s_A^{\left(n\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\alpha \right], b_A^{\left(n\right)}\left[a^{\left(n\right)}\left(j_{n-1}\dots j_1;x\right);\alpha \right]=0$$

(3)

Concatenating Equations (2) and (3) yields the following (n + 1)^th-Level Forward Sensitivity System [abbreviated as (n + 1)^th-LFSS] for the function $u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)={\left[u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);\right]}^{†}$:

$${\left\{F^{\left(n+1\right)}\left(\alpha ;x\right)u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)\right\}}_{{\alpha }^0}= {\left\{q_F^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n\right)};x;\alpha \right)\right\}}_{{\alpha }^0}, x\in {\mathsf{\Omega }}_x \left({\alpha }^0\right)$$

(4)

$${\left\{ b_F^{\left(n+1\right)}\left[u^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right); \alpha \right]\right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(5)

where:

$$\begin{array}{cc}\hfill & {\mathbf{F}}^{\left(n+1\right)}\left(\mathsf{\alpha };\mathbf{x}\right)\triangleq \left(\begin{array}{cc}{\left[{\mathbf{F}}^{\left(n\right)}\right]}_{2^n\times 2^n}& {\left[\mathbf{0}\right]}_{2^n\times 2^n}\\ {\left[\mathbf{0}\right]}_{2^n\times 2^n}& {\left[{\mathbf{A}}^{\left(n\right)}\right]}_{2^n\times 2^n}\end{array}\right);{\mathbf{u}}^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;\mathbf{x}\right)\triangleq \left(\begin{array}{c}{\mathbf{u}}^{\left(n\right)}\left(j_{n-2},\dots ,j_1\right)\\ {\mathbf{a}}^{\left(n\right)}\left(j_{n-1},\dots ,j_1\right)\end{array}\right);\hfill \\ \hfill & {\mathbf{q}}_F^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;\mathbf{x};\mathsf{\alpha }\right)\triangleq \left(\begin{array}{c}{\mathbf{q}}_F^{\left(n\right)}\\ {\mathbf{s}}_A^{\left(n\right)}\end{array}\right);{\mathbf{b}}_F^{\left(n+1\right)}\left[{\mathbf{u}}^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \left(\begin{array}{c}{\mathbf{b}}_F^{\left(n\right)}\\ {\mathbf{b}}_A^{\left(n\right)}\end{array}\right).\hfill \end{array}$$

(6)

In terms of the function $u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)\triangleq {\left[u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right)\right]}^{†}$, each n^th-order sensitivity can be written as follows:

$$\frac{{\partial }^{n}R\left(\phi ;\psi ;\alpha \right)}{\partial {\alpha }_{j_1}\dots \partial {\alpha }_{j_n}}\equiv R^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)\equiv {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}}.$$

(7)

The total G-differential of $R^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)$ is obtained, by definition, as follows:

$$\begin{array}{cc}\hfill & {\left\{\delta R^{\left(n\right)}\left(j_n,\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)};\mathsf{\alpha };\delta {\mathbf{u}}^{\left(n+1\right)};\delta \mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ \hfill & \triangleq {\left\{\frac{d}{d\epsilon }{\left[\prod _{i=1}^{TI}\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(n\right)}\left(j_n,\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)}+\epsilon \delta {\mathbf{u}}^{\left(n+1\right)};\mathsf{\alpha }+\epsilon \delta \mathsf{\alpha }\right)\right]}_{{\mathsf{\alpha }}^0}\right\}}_{\epsilon =0}\hfill \\ \hfill & ={\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right)}_{{\mathsf{\alpha }}^0}\right\}}_{dir}+{\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)};\mathsf{\alpha };\delta {\mathbf{u}}^{\left(n+1\right)}\right)}_{{\mathsf{\alpha }}^0}\right\}}_{ind},\hfill \end{array}.$$

(8)

where the direct effect term ${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}$ depends directly on the vector of parameter variations $\delta \alpha$, and is defined as follows, for $j_1=1,\dots ,TP; j_2=1,\dots ,j_1;\dots ;j_n=1,\dots ,j_{n-1}$:

$${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{\frac{\partial }{\partial \alpha }{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}}\right\}}_{{\alpha }^0}\delta \alpha ,$$

(9)

and where the indirect-effect term ${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ is defined as follows:

$${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\triangleq {\left\{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i\frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial u^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)}}}\right\}}_{{\alpha }^0}\delta u^{\left(n+1\right)},$$

(10)

with

$$\begin{array}{l}\frac{\partial \left[ \right]}{\partial u^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)}\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)\equiv \frac{\partial \left[ \right]}{\partial \phi }\delta \phi \left(x\right)+\frac{\partial \left[ \right]}{\partial \psi }\delta \psi \left(x\right)+{\displaystyle \sum _{k=1}^2\frac{\partial \left[ \right]}{\partial a_k^{\left(1\right)}}}\delta a_k^{\left(1\right)}\left(x\right)\\ +{\displaystyle \sum _{k=1}^{2^2}\frac{\partial \left[ \right]}{\partial a_k^{\left(2\right)}\left(j_1\right)}\delta a_k^{\left(2\right)}\left(j_1;x\right)+\dots +{\displaystyle \sum _{k=1}^{2^n}\frac{\partial \left[ \right]}{\partial a_k^{\left(n\right)}\left(j_{n-1},\dots ,j_1\right)}\delta a_k^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right)},}\end{array}$$

(11)

The vector of variations $\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)\triangleq {\left[\delta u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\delta a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right)\right]}^{†}$ is the solution of the G-differentiated (n + 1)^th-LFS represented by Equations (4) and (5), which is equivalent concatenating the n^th-LVSS to the system of equations obtained by G-differentiating the n^th-LASS while being subject to the corresponding boundary conditions. The n^th-LVSS is assumed to be correct as given in Table 5, namely:

$${\left[V^{\left(n\right)}\right]}_{2^n\times 2^n}\delta u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right)=q_V^{\left(n\right)}\left[j_{n-2},\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\alpha ;\delta \alpha \right]$$

(12)

$$b_V^{\left(n\right)} \triangleq {\left[b_V^{\left(n-1\right)},\delta b_A^{\left(n-1\right)}\right]}^{†} ={\left[0,0\right]}^{†}$$

(13)

The G-differential of the n^th-LASS is obtained from Equation (3), in the following form:

$${\left[A^{\left(n\right)} \right]}_{2^n\times 2^n} \delta a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right) =p^{\left(n\right)}\left(j_{n-1}, ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right),$$

(14)

$$\delta b_A^{\left(n\right)}\left[a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\alpha \right]=0$$

(15)

where

$$p^{\left(n\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right) \triangleq \frac{\partial \left\{s_A^{\left(n\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\alpha \right]-A^{\left(n\right)}{a}^{\left(n\right)}\right\}}{\partial \alpha }\delta \alpha .$$

(16)

Concatenating Equations (12)−(15) yields the following (n + 1)^th-Level Variational Sensitivity System [(n + 1)^th-LVSS] for the function $\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)$:

$${\left[V^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)=q_V^{\left(n+1\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha ;\delta \alpha \right],$$

(17)

$$b_V^{\left(n+1\right)}\left(\delta u^{\left(n+1\right)};\delta \alpha \right)\triangleq \left(\begin{array}{c}{b}_V^{\left(n\right)}\left(\delta u^{\left(n\right)};\delta \alpha \right)\\ \delta b_A^{\left(n\right)}\left(\delta a^{\left(n\right)} ;\delta \alpha \right)\end{array}\right)=\left(\begin{array}{c}\left[0\right]\\ \left[0\right]\end{array}\right), x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(18)

where:

$$\begin{array}{cc}\hfill & {\left[{\mathbf{V}}^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}\triangleq {\left(\begin{array}{cc}{\left[{\mathbf{V}}^{\left(n\right)}\right]}_{2^n\times 2^n}& {\left[\mathbf{0}\right]}_{2^n\times 2^n}\\ {\left[{\mathbf{V}}_{21}^{\left(n+1\right)}\right]}_{2^n\times 2^n}& {\left[{\mathbf{V}}_{22}^{\left(n+1\right)}\right]}_{2^n\times 2^n}\end{array}\right)}_{2^{n+1}\times 2^{n+1}};{\mathbf{V}}_{22}^{\left(n\right)}\left(j_1\right)\triangleq {\mathbf{A}}^{\left(n\right)};\hfill \\ \hfill & {\mathbf{V}}_{21}^{\left(n+1\right)}\triangleq \frac{\partial \left\{{\left[{\mathbf{A}}^{\left(n\right)}\right]}_{2^n\times 2^n}{\mathbf{a}}^{\left(n\right)}\left(j_{n-1},\dots ,j_1;\mathbf{x}\right)\right\}}{\partial {\mathbf{u}}^{\left(n\right)}}-\frac{\partial \left\{{\mathbf{s}}_A^{\left(n\right)}\left[j_{n-1},\dots ,j_1;{\mathbf{u}}^{\left(n\right)}\left(j_{n-2},\dots ,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}{\partial {\mathbf{u}}^{\left(n\right)}};\hfill \end{array}$$

(19)

$$\begin{array}{l}{q}_V^{\left(n+1\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right]\triangleq \left(\begin{array}{c}{q}_V^{\left(n\right)}\left[j_{n-2},\dots ,j_1;\alpha ;\delta \alpha \right]\\ p^{\left(n\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right]\end{array}\right)\\ \equiv {\left[.....q_k^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right)\dots \right]}^{†},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^{n+1}.\end{array}$$

(20)

In principle, the (n + 1)^th-LVSS, comprising Equations (17) and (18), could be solved to determine the variational vector $\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)$ which could be used, in turn, to determine the indirect-effect term ${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$. On the other hand, the need for solving the (n + 1)^th-LVSS can be avoided by expressing the indirect-effect term ${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ in an alternative way, which does not involve the function $\delta u^{\left(n+1\right)}\left(j_{n-1}\dots j_1;x\right)$. This alternative expression will be obtained by using the solution of a (n + 1)^th-Level Adjoint Sensitivity Systems [(n + 1)^th-LASS], which will be constructed below by implementing the same sequence of logical steps as were followed for constructing the 1^st-LASS (and all of the other lower-level adjoint sensitivity systems).

The (n + 1)^th-LASS is constructed in a Hilbert space, denoted as ${\mathrm{H}}_{n+1}$, comprising block-vector elements of the form ${\eta }^{\left(n+1\right)}\left(x\right)\in {\mathrm{H}}_{n+1}$,${\eta }^{\left(n+1\right)}\left(x\right)\triangleq {\left[\dots ,{\eta }_k^{\left(n+1\right)}\left(x\right),\dots \right]}^{†};\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^{n+1}$ with elements having the following structure: ${\eta }_k^{\left(n+1\right)}\left(x\right)\triangleq {\left[{\eta }_{k,1}^{\left(n+1\right)}\left(x\right),\dots ,{\eta }_{k,TD}^{\left(n+1\right)}\left(x\right)\right]}^{†}$. The inner product between two elements, ${\eta }^{\left(n+1\right)}\left(x\right)\in {\mathrm{H}}_{n+1}$ and ${\xi }^{\left(n+1\right)}\left(x\right)\in {\mathrm{H}}_{n+1}$, of the Hilbert space ${\mathrm{H}}_{n+1}$, will be denoted as ${〈{\eta }^{\left(n+1\right)}\left(x\right),{\xi }^{\left(n+1\right)}\left(x\right)〉}_{n+1}$ and is defined as follows:

$${〈{\eta }^{\left(n+1\right)}\left(x\right),{\xi }^{\left(n+1\right)}\left(x\right)〉}_{n+1}\triangleq {\displaystyle \sum _{k=1}^{2^{n+1}}{〈{\eta }_k^{\left(n+1\right)}\left(x\right),{\xi }_k^{\left(n+1\right)}\left(x\right)〉}_0}.$$

(21)

Using the definition provided in Equation (21), form the inner product in ${\mathrm{H}}_{n+1}$ of Equation (12) with a yet undefined vector-valued function $a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)\triangleq {\left[\dots ,a_k^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right),\dots \right]}^{†}\in {\mathrm{H}}_{n+1}; k=1,\dots ,2^{n+1}$, $j_1=1,\dots ,TP$, $j_2=1,\dots ,j_1$; $j_{n+1}=1,\dots ,j_n$, to obtain the following relation:

$$\begin{array}{cc}\hfill & {\left\{{〈{\mathbf{a}}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\mathbf{x}\right),\left[{\mathbf{V}}^{\left(n+1\right)}\right]\delta {\mathbf{u}}^{\left(n+1\right)}〉}_{n+1}\right\}}_{{\mathsf{\alpha }}^0}=,\hfill \\ \hfill & {\left\{{〈{\mathbf{a}}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\mathbf{x}\right),{\mathbf{q}}_V^{\left(n+1\right)}\left[j_{n-1}\dots j_1;{\mathbf{u}}^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]〉}_{n+1}\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ \hfill & ={\left\{{〈\delta {\mathbf{u}}^{\left(n+1\right)},{\mathbf{A}}^{\left(n+1\right)}{\mathbf{a}}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\mathbf{x}\right)〉}_{n+1}\right\}}_{{\mathsf{\alpha }}^0}+{\left\{P^{\left(n+1\right)}\left[\delta {\mathbf{u}}^{\left(n+1\right)};{\mathbf{a}}^{\left(n+1\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0},\hfill \end{array}$$

(22)

where ${\left\{P^{\left(n+1\right)}\left[\delta u^{\left(n+1\right)};a^{\left(n+1\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$ denotes the bilinear concomitant defined on the phase-space boundary $x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right)$, evaluated at the nominal values of the model parameter and respective functions, and where:

$${\left[A^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}\triangleq {\left[V^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}^{*}={\left(\begin{array}{cc}{\left[V^{\left(n\right)}\right]}_{2^n\times 2^n}& {\left[0\right]}_{2^n\times 2^n}\\ {\left[V_{21}^{\left(n+1\right)}\right]}_{2^n\times 2^n}& {\left[V_{22}^{\left(n+1\right)}\right]}_{2^n\times 2^n}\end{array}\right)}^{*}=\left(\begin{array}{cc}{A}^{\left(n+1\right)}& {\left[V_{21}^{\left(n+1\right)}\right]}^{*}\\ {\left[0\right]}_{2^n\times 2^n}& {\left[V_{22}^{\left(n+1\right)}\right]}^{*}\end{array}\right)$$

(23)

The first term on right-side of the second equality in Equation (22) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$, which is achieved by requiring that the (n + 1)^th-level adjoint function $a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)\triangleq {\left[\dots ,a_k^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right),\dots \right]}^{†}\in {\mathrm{H}}_{n+1};\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^{n+1}$, be the solution of the following (n + 1)^th-Level Adjoint Sensitivity System, for $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$;…$j_n=1,\dots ,j_{n-1}$:

$${\left[A^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}{a}^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right) =s_A^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right),$$

(24)

$${\left\{b_A^{\left(n+1\right)}\left[a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right);u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right).$$

(25)

where the vector $s_A^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq {\left[\dots ,s_k^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right),\dots \right]}^{†}$, $k=1,\dots ,2^{n+1}$, comprises $2^{n+1}$ components defined as follows, for each $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$;…;$j_n=1,\dots ,j_{n-1}$:

$$s_1^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial \phi },$$

(26)

$$s_2^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial \psi },$$

(27)

$$s_{2+k}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial a_k^{\left(1\right)}}; k=1,2;$$

(28)

$$s_{2^2+k}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial a_k^{\left(2\right)}\left(j_1\right)}; k=1,‥,2^2;$$

(29)

$$s_{2^3+k}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial a_k^{\left(3\right)}\left(j_2;j_1\right)}; k=1,‥,2^3 ;$$

(30)

$$s_{2^{n-1}+k}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial a_k^{\left(n-1\right)}\left(j_{n-2},\dots ,j_1\right)}; k=1,‥,2^{n-1};$$

(31)

$$s_{2^n+k}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(n\right)}\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha \right)}{\partial a_k^{\left(n\right)}\left(j_{n-1},\dots ,j_1\right)}; k=1,‥,2^n.$$

(32)

The (n + 1)^th-level adjoint boundary conditions represented by Equation (25) are selected so as to eliminate, in conjunction with the boundary conditions represented by Equation (18), all of the unknown values of the functions $\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)$ in the expression of the bilinear concomitant ${\left\{P^{\left(n+1\right)}\left[\delta u^{\left(n+1\right)};a^{\left(n+1\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$. This bilinear concomitant may vanish after implementing the boundary conditions represented by Equations (18) and (25); if it does not vanish, this bilinear concomitant will be reduced to a residual quantity which will comprise only known values of $a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)$, $u^{\left(n+1\right)}\left(j_{n-1}\dots j_1;x\right)$, $\alpha$ and $\delta \alpha$, and will be denoted as ${\left\{{\widehat{P}}^{\left(n+1\right)}\left(a^{\left(n+1\right)};u^{\left(n+1\right)};\alpha ;\delta \alpha \right)\right\}}_{{\alpha }^0}$.

Using the equations underlying the 4^th-LASS together with the relation provided in Equation in Equation yields the following expression for the indirect-effect term ${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ in terms of the (n + 1)^th-level adjoint functions $a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)$, for each $j_1=1,\dots ,TP\hspace{0.17em}$; $j_2=1,\dots ,j_1$;…;$j_n=1,\dots ,j_{n-1}$:

$$\begin{array}{l}{\left\{\delta R^{\left(n\right)}{\left[j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right]}_{{\alpha }^0}\right\}}_{ind}=-{\left\{{\widehat{P}}^{\left(n+1\right)}\left(a^{\left(n+1\right)};u^{\left(n+1\right)};\alpha ;\delta \alpha \right)\right\}}_{{\alpha }^0}\\ +{\left\{{〈a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right),q_V^{\left(n+1\right)}\left[j_{n-1},\dots ,j_1;u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha ;\delta \alpha \right] 〉}_{n+1}\right\}}_{{\alpha }^0}.\end{array}$$

(33)

Adding the result obtained in Equation (33) for the indirect effect term to the result provided in Equation (9) for the direct effect term yields the following expression for the total n^th-order G-variation (of the response) defined in Equation (8):

$$\begin{array}{cc}\hfill & {\left\{\delta R^{\left(n\right)}\left(j_n,\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)};\mathsf{\alpha };\delta {\mathbf{u}}^{\left(n+1\right)};\delta \mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}={\left\{\frac{\partial }{\partial \mathsf{\alpha }}\prod _{i=1}^{TI}\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(n\right)}\left(j_n,\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)};\mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}\delta \mathsf{\alpha },\hfill \\ \hfill & +{\left\{{〈{\mathbf{a}}^{\left(n+1\right)}\left(j_n,\dots ,j_1;\mathbf{x}\right),{\mathbf{q}}_V^{\left(n+1\right)}\left[j_{n-1},\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]〉}_{n+1}\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ \hfill & -{\left\{{\widehat{P}}^{\left(n+1\right)}\left({\mathbf{a}}^{\left(n+1\right)};{\mathbf{u}}^{\left(n+1\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}\equiv \sum _{j_{n+1}=1}^{TP}{\left\{R^{\left(n+1\right)}\left(j_{n+1},\dots ,j_1;{\mathbf{u}}^{\left(n+1\right)};{\mathbf{a}}^{\left(n+1\right)};\mathsf{\alpha }\right)\right\}}_{\left({\mathsf{\alpha }}^0\right)}\delta {\alpha }_{j_{n+1}}.\hfill \end{array}$$

(34)

where $R^{\left(n+1\right)}\left(j_{n+1},\dots ,j_1;u^{\left(n+1\right)};a^{\left(n+1\right)};\alpha \right)$ denotes the (n + 1)^th-order partial sensitivity of the response $R\left[u^{\left(1\right)}\left(x\right);\alpha \right]$ with respect to the model parameters, evaluated at the nominal parameter values ${\alpha }^0$. The results obtained in this Subsection are summarized in

Table 6. (n + 1)^th-Order Sensitivities.

(n + 1)th-Level Forward System	${\mathrm{n}}^{\mathrm{th}}-\mathrm{Level} \mathrm{Forward} \mathrm{System}+{\mathrm{n}}^{\mathrm{th}}-\mathrm{LASS}: F^{\left(n+1\right)}{u}^{\left(n+1\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(n+1\right)}$; $F^{\left(n+1\right)}\left(\alpha ;x\right)\triangleq diag\left(F^{\left(n\right)},A^{\left(n\right)}\right)$$; \hspace{0.17em}{b}_F^{\left(n+1\right)}\left(u^{\left(n+1\right)};\hspace{0.17em}\alpha \right)\triangleq {\left(\hspace{0.17em}{b}_F^{\left(n\right)},b_A^{\left(n\right)}\right)}^{†}={\left[0,0\right]}^{†}$ $u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)={\left[u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right),a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right)\right]}^{†}$
(n + 1)th-LVSS	${\left[V^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)=q_V^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta \alpha \right)$ $\delta u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right)={\left[\delta u^{\left(n\right)}\left(j_{n-2},\dots ,j_1;x\right);\delta a^{\left(n\right)}\left(j_{n-1},\dots ,j_1;x\right);\right]}^{†}$ $b_V^{\left(n+1\right)}\hspace{0.17em}\triangleq {\left[b_V^{\left(n\right)},\delta b_A^{\left(n\right)}\right]}^{†}\hspace{0.17em}={\left[0,0\right]}^{†}$$; {\left[V^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}\triangleq \left(\begin{array}{cc}{V}^{\left(n\right)}& {\left[0\right]}_{2^n\times 2^n}\\ V_{21}^{\left(n+1\right)}& V_{22}^{\left(n+1\right)}\end{array}\right)$
(n + 1)th-Level Hilbert Space; Inner Product	${\mathsf{H}}_n$$; {〈{\eta }^{\left(n+1\right)}\left(x\right),{\xi }^{\left(n+1\right)}\left(x\right)〉}_n\triangleq {\displaystyle \sum _{i=1}^{2^{n+1}}{〈{\eta }_i^{\left(n+1\right)}\left(x\right),{\xi }_i^{\left(n+1\right)}\left(x\right)〉}_0}.$
(n + 1)th-LASS State Function	$a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)\triangleq \left[\dots ,a_k^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right),\dots \right];\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^{n+1}$
(n + 1)th-LASS	${\left[A^{\left(n+1\right)}\hspace{0.17em}\right]}_{2^{n+1}\times 2^{n+1}}\hspace{0.17em}{a}^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right)\hspace{0.17em}=s_A^{\left(n+1\right)}\left[j_n,\dots ,j_1;u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha \right]\hspace{0.17em}$; ${\left[A^{\left(n+1\right)}\hspace{0.17em}\right]}_{2^{n+1}\times 2^{n+1}}\hspace{0.17em}\triangleq {\left[V^{\left(n+1\right)}\right]}_{2^{n+1}\times 2^{n+1}}^{*}$$; b_A^{\left(n+1\right)}\left[a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right);u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right);\alpha \right]=0$
No. of Distinct Sensitivities	$TP\left(TP+1\right)\left(TP+2\right)\dots \left(TP+n\right)/\left(n+1\right)!$$\begin{array}{l}{R}^{\left(n+1\right)}\left[j_{n+1},\dots ,j_1;u^{\left(n+1\right)}\left(j_{n-1},\dots ,j_1;x\right);a^{\left(n+1\right)}\left(j_n,\dots ,j_1;x\right);\alpha \right]\\ \triangleq {\partial }^{n+1}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\dots \partial {\alpha }_{j_{n+1}};j_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}{j}_2=1,\dots ,j_1;.....j_{n+1}=1,\dots ,j_n\end{array}$

.

The results summarized in Table 6 were obtained by applying the general principles underlying adjoint sensitivity analysis methodology to the n^th-order sensitivities, considered as “model responses.” The expression of the (n + 1)^th-order sensitivities were obtained by determining the first-order differential of the n^th-order sensitivities, which yields an expression that is identical to the expression that would be obtained by advancing the index, from n to (n + 1), of the expressions of the n^th-order sensitivities. Thus, the proof by induction of the general framework is thereby completed.

3. Particular Case: The 5^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Coupled Forward/Adjoint Linear Systems (5^th-CASAM-L)

As an additional application of the general framework of the n^th-CASAM, this Subsection will present the “5^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Coupled Forward/Adjoint Linear Systems” (5^th-CASAM-L). The derivation of the 5^th-CASAM-L will be performed explicitly, by starting with the results produced by the 4^th-CASAM-L, which are presented in Section 5 and summarized in Table 4. It will be shown that the results thus obtained for the 5^th-CASAM-L coincide with the results that would be predicted by the n^th-CASAM-L (Table 5) for the particular value n = 5.

The 5^th-CASAM-L commences by recalling the end-products of the 4^th-CASAM-L, namely the expressions of the 4^th-order partial sensitivity ${\left\{R^{\left(4\right)}\left[j_4,j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]\right\}}_{\left({\alpha }^0\right)}$ of the response with respect to the model parameters, which was obtained in Equation (203) in Section 5, and which can be written in the following form:

$$\begin{array}{l}{\left\{R^{\left(4\right)}\left[j_4,j_3,j_2,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]\right\}}_{\left({\alpha }^0\right)}\triangleq {\partial }^{4}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\dots \partial {\alpha }_{j_4}\\ \triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(4\right)}\left[j_4,\dots ,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]}} .\end{array}$$

(35)

The function $u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)\triangleq {\left[u^{\left(4\right)}\left(j_2,j_1;x\right), a^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\right]}^{†}$ is the solution of the following 5^th-Level Forward System (5^th-LFS), which is obtained by concatenating Equations (179), (180), (195) and (196) from Section 5, which are written in block-matrix form as follows:

$${\left\{F^{\left(5\right)}\left(\alpha ;x\right)u^{\left(5\right)}\left(j_3,j_3,j_1;x\right)\right\}}_{{\alpha }^0}= {\left\{q_F^{\left(5\right)}\left(j_4,\dots ,j_1;u^{\left(4\right)};x;\alpha \right)\right\}}_{{\alpha }^0}, x\in {\mathsf{\Omega }}_x \left({\alpha }^0\right)$$

(36)

$${\left\{ b_F^{\left(5\right)}\left[u^{\left(5\right)}\left(j_4,\dots ,j_1;x\right); \alpha \right]\right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(37)

where:

$$\begin{array}{l}{F}^{\left(5\right)}\left(\alpha ;x\right)\triangleq \left(\begin{array}{cc}{\left[F^{\left(4\right)}\right]}_{2^4\times 2^4}& {\left[0\right]}_{2^4\times 2^4}\\ {\left[0\right]}_{2^4\times 2^4}& {\left[A^{\left(4\right)}\right]}_{2^4\times 2^4}\end{array}\right); u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)\triangleq \left(\begin{array}{c}{u}^{\left(4\right)}\left(j_2,j_1\right)\\ a^{\left(4\right)}\left(j_3,j_2,j_1\right)\end{array}\right);\\ q_F^{\left(5\right)}\left(j_4,\dots ,j_1;x;\alpha \right)\triangleq \left(\begin{array}{c}{q}_F^{\left(4\right)}\\ s_A^{\left(4\right)}\end{array}\right); b_F^{\left(5\right)}\left[u^{\left(5\right)}\left(j_3,j_2,j_1;x\right); \alpha \right]\triangleq \left(\begin{array}{c} b_F^{\left(4\right)}\\ b_A^{\left(4\right)}\end{array}\right).\end{array}$$

(38)

The 5^th-order total differential of the response $R\left[u^{\left(1\right)}\left(x\right);\alpha \right]$ is obtained by G-differentiating the 4^th-order total G-differential of the response, to obtain the following expression:

$$\begin{array}{l}{\left\{\delta R^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)};\delta \alpha \right)\right\}}_{{\alpha }^0}\\ \triangleq {\left\{\frac{d}{d\epsilon }{\left[{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(4\right)}\left(j_4,\dots ,j_1 ;u^{\left(5\right)}+\epsilon \delta u^{\left(5\right)};\alpha +\epsilon \delta \alpha \right)}}\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}\\ ={\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)}\right)}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(39)

where the direct effect term ${\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}$ depends directly on the vector of parameter variations $\delta \alpha$, and is defined as follows, for $j_1=1,\dots ,TP; j_2=1,\dots ,j_1;\dots ;j_4=1,\dots ,j_3$:

$${\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta \alpha \right)}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{\frac{\partial }{\partial \alpha }{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}}\right\}}_{{\alpha }^0}\delta \alpha ,$$

(40)

and where the indirect-effect term ${\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ is defined as follows:

$${\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)}\right)}_{{\alpha }^0}\right\}}_{ind}\triangleq {\left\{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i\frac{\partial S^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}{\partial u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)}}}\right\}}_{{\alpha }^0}\delta u^{\left(5\right)},$$

(41)

with

$$\begin{array}{l}\frac{\partial \left[ \right]}{\partial u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)}\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)\equiv \frac{\partial \left[ \right]}{\partial \phi }\delta \phi \left(x\right)+\frac{\partial \left[ \right]}{\partial \psi }\delta \psi \left(x\right)+{\displaystyle \sum _{k=1}^2\frac{\partial \left[ \right]}{\partial a_k^{\left(1\right)}}}\delta a_k^{\left(1\right)}\left(x\right)\\ +{\displaystyle \sum _{k=1}^{2^2}\frac{\partial \left[ \right]}{\partial a_k^{\left(2\right)}\left(j_1\right)}\delta a_k^{\left(2\right)}\left(j_1;x\right)+{\displaystyle \sum _{k=1}^{2^3}\frac{\partial \left[ \right]}{\partial a_k^{\left(3\right)}\left(j_2,j_1\right)}\delta a_k^{\left(3\right)}\left(j_2,j_1;x\right)}+{\displaystyle \sum _{k=1}^{2^4}\frac{\partial \left[ \right]}{\partial a_k^{\left(4\right)}\left(j_3,j_2,j_1\right)}\delta a_k^{\left(4\right)}\left(j_3,j_2,j_1;x\right)},}\end{array}$$

(42)

The vector of variations $\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)\triangleq {\left[\delta u^{\left(4\right)}\left(j_2,j_1;x\right);\delta a^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\right]}^{†}$ is the solution of the G-differentiated 5^th-LFS represented by Equations (36) and (37), which is equivalent concatenating the 4^th-LVSS to the system of equations obtained by G-differentiating the 4^th-LASS while being subject to the corresponding boundary conditions. The 4^th-LVSS has been determined in Section 5 and is provided in Equations (186) and (187). On the other hand, the G-differential of the 4^th-LASS is obtained from Equations (195) and (196), and has the following form:

$${\left[A^{\left(4\right)} \right]}_{2^4\times 2^4} \delta a^{\left(4\right)}\left(j_3,j_2,j_1;x\right) =p^{\left(4\right)}\left(j_3,j_2,j_1;u^{\left(5\right)};\alpha ;\delta \alpha \right),$$

(43)

$$\delta b_A^{\left(4\right)}\left[a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]=0$$

(44)

where

$$p^{\left(4\right)}\left(j_3,j_2,j_1;u^{\left(5\right)};\alpha ;\delta \alpha \right) \triangleq \frac{\partial \left\{s_A^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]-A^{\left(4\right)}{a}^{\left(4\right)}\right\}}{\partial \alpha }\delta \alpha .$$

(45)

Concatenating the 4^th-LVSS with the 4^th-LASS yields the following 5^th-Level Variational Sensitivity System (5^th-LVSS) for the function $\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)$:

$${\left[V^{\left(5\right)}\right]}_{2^5\times 2^5}\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)=q_V^{\left(5\right)}\left[j_3,j_2,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha ;\delta \alpha \right],$$

(46)

$$b_V^{\left(5\right)}\left(\delta u^{\left(5\right)};\delta \alpha \right)\triangleq \left(\begin{array}{c}{b}_V^{\left(4\right)}\left(\delta u^{\left(4\right)};\delta \alpha \right)\\ \delta b_A^{\left(4\right)}\left(\delta a^{\left(4\right)} ;\delta \alpha \right)\end{array}\right)=\left(\begin{array}{c}\left[0\right]\\ \left[0\right]\end{array}\right), x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(47)

where:

$$\begin{array}{l}{\left[V^{\left(5\right)}\right]}_{2^5\times 2^5}\triangleq \left(\begin{array}{cc}{\left[V^{\left(4\right)}\right]}_{2^4\times 2^4}& {\left[0\right]}_{2^4\times 2^4}\\ {\left[V_{21}^{\left(5\right)}\right]}_{2^4\times 2^4}& {\left[V_{22}^{\left(5\right)}\right]}_{2^4\times 2^4}\end{array}\right); V_{22}^{\left(4\right)}\triangleq A^{\left(4\right)}; V_{21}^{\left(5\right)}\triangleq \frac{\partial \left\{A^{\left(4\right)}{a}^{\left(4\right)}-s_A^{\left(4\right)}\right\}}{\partial u^{\left(4\right)}};\\ \end{array}$$

(48)

$$q_V^{\left(5\right)}\left[j_3,j_3,j_1;u^{\left(5\right)};\alpha ;\delta \alpha \right]\triangleq \left(\begin{array}{c}{q}_V^{\left(4\right)}\\ p^{\left(4\right)}\end{array}\right)\equiv {\left[.....q_k^{\left(5\right)}\left(j_3,j_2,j_1;u^{\left(5\right)};\alpha ;\delta \alpha \right)\dots \right]}^{†}, k=1,\dots ,2^5.$$

(49)

In principle, the 5^th-LVSS, comprising Equations (46) and (47), could be solved to determine the variational vector $\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)$ which could be used, in turn, to determine the indirect-effect term ${\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$. However, the need for solving the 5^th-LVSS can be avoided by expressing the indirect-effect term ${\left\{\delta R^{\left(n\right)}{\left(j_n,\dots ,j_1;u^{\left(n+1\right)};\alpha ;\delta u^{\left(n+1\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ in terms of the solution of a 5^th-Level Adjoint Sensitivity Systems (5^th-LASS), which is constructed in a Hilbert space, denoted as ${\mathrm{H}}_5$, comprising block-vector elements of the form ${\eta }^{\left(5\right)}\left(x\right)\in {\mathrm{H}}_5$,${\eta }^{\left(5\right)}\left(x\right)\triangleq {\left[\dots ,{\eta }_k^{\left(5\right)}\left(x\right),\dots \right]}^{†}; k=1,\dots ,2^5$, with elements having the following structure: ${\eta }_k^{\left(5\right)}\left(x\right)\triangleq {\left[{\mathsf{\eta }}_{k,1}^{\left(5\right)}\left(x\right),\dots ,{\mathsf{\eta }}_{k,TD}^{\left(5\right)}\left(x\right)\right]}^{†}$. The inner product between two elements, ${\eta }^{\left(5\right)}\left(x\right)\in {\mathrm{H}}_5$ and ${\xi }^{\left(5\right)}\left(x\right)\in {\mathrm{H}}_5$ will be denoted as ${〈{\eta }^{\left(5\right)}\left(x\right),{\xi }^{\left(5\right)}\left(x\right)〉}_5$ and is defined as follows:

$${〈{\eta }^{\left(5\right)}\left(x\right),{\xi }^{\left(5\right)}\left(x\right)〉}_5\triangleq {\displaystyle \sum _{k=1}^{2^5}{〈{\eta }_k^{\left(5\right)}\left(x\right),{\xi }_k^{\left(5\right)}\left(x\right)〉}_0}.$$

(50)

Using the definition provided in Equation (50), form the inner product in ${\mathrm{H}}_5$ of Equation (46) with a yet undefined vector-valued function $a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right)\triangleq {\left[\dots ,a_k^{\left(5\right)}\left(j_4,\dots ,j_1;x\right),\dots \right]}^{†}\in {\mathrm{H}}_5; k=1,\dots ,2^5$,$j_1=1,\dots ,TP$, $j_2=1,\dots ,j_1$;…;$j_5=1,\dots ,j_5$, to obtain the following relation:

$$\begin{array}{l}{\left\{{〈a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right),V^{\left(5\right)}\delta u^{\left(5\right)}〉}_5\right\}}_{{\alpha }^0}={\left\{{〈a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right),q_V^{\left(5\right)}\left[j_3,j_2,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha ;\delta \alpha \right] 〉}_5\right\}}_{{\alpha }^0}\\ ={\left\{{〈\delta u^{\left(5\right)}, A^{\left(5\right)}{a}^{\left(5\right)}\left(j_4,\dots ,j_1;x\right) 〉}_5\right\}}_{{\alpha }^0}+{\left\{P^{\left(5\right)}\left[\delta u^{\left(5\right)};a^{\left(5\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0},\end{array}$$

(51)

where ${\left\{P^{\left(5\right)}\left[\delta u^{\left(5\right)};a^{\left(5\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$ denotes the bilinear concomitant defined on the phase-space boundary $x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right)$, evaluated at the nominal values of the model parameter and respective functions, and where:

$${\left[A^{\left(5\right)}\right]}_{2^5\times 2^5}\triangleq {\left[V^{\left(5\right)}\right]}_{2^5\times 2^5}^{*}={\left(\begin{array}{cc}{\left[V^{\left(4\right)}\right]}_{2^4\times 2^4}& {\left[0\right]}_{ 2^4\times 2^4}\\ {\left[V_{21}^{\left(5\right)}\right]}_{2^4\times 2^4}& {\left[V_{22}^{\left(5\right)}\right]}_{2^4\times 2^4}\end{array}\right)}^{*}=\left(\begin{array}{cc}{A}^{\left(5\right)}& {\left[V_{21}^{\left(5\right)}\right]}^{*}\\ {\left[0\right]}_{2^4\times 2^4}& {\left[V_{22}^{\left(5\right)}\right]}^{*}\end{array}\right)$$

(52)

The first term on right-side of the second equality in Equation (51) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$, which is achieved by requiring that the 5^th-level adjoint function $a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right)\triangleq {\left[\dots ,a_k^{\left(5\right)}\left(j_4,\dots ,j_1;x\right),\dots \right]}^{†}\in {\mathrm{H}}_5; k=1,\dots ,2^5$, be the solution of the following 5^th-Level Adjoint Sensitivity System (5^th-LASS), for $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$;…;$j_4=1,\dots ,j_3$:

$${\left[A^{\left(5\right)}\right]}_{2^5\times 2^5}{a}^{\left(5\right)}\left(j_4,\dots ,j_1;x\right) =s_A^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right) ,$$

(53)

$${\left\{b_A^{\left(5\right)}\left[a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right);u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right).$$

(54)

where the vector $s_A^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right)\triangleq {\left[\dots ,s_k^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right),\dots \right]}^{†}$, $k=1,\dots ,2^5$, comprises $2^5$ components defined as follows, for each $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$;…;$j_4=1,\dots ,j_3$:

$$s_1^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}{\partial \phi },$$

(55)

$$s_2^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}{\partial \psi },$$

(56)

$$s_{2+k}^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right)\triangleq \frac{\partial S^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}{\partial a_k^{\left(1\right)}}; k=1,2;$$

(57)

$$s_{2^2+k}^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right) \triangleq \frac{\partial S^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}{\partial a_k^{\left(2\right)}\left(j_1\right)}; k=1,‥,2^2;$$

(58)

$$s_{2^3+k}^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right) \triangleq \frac{\partial S^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}{\partial a_k^{\left(3\right)}\left(j_2;j_1\right)}; k=1,‥,2^3 ;$$

(59)

$$s_{2^4+k}^{\left(5\right)}\left(j_4,\dots ,j_1;\alpha \right) \triangleq \frac{\partial S^{\left(4\right)}\left(j_n,\dots ,j_1;u^{\left(5\right)};\alpha \right)}{\partial a_k^{\left(4\right)}\left(j_3,j_3,j_1\right)}; k=1,‥,2^4.$$

(60)

The 5^th-level adjoint boundary conditions represented by Equation (54) are selected so as to eliminate, in conjunction with the boundary conditions represented by Equation (47), all of the unknown values of the functions $\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)$ in the expression of ${\left\{P^{\left(5\right)}\left[\delta u^{\left(5\right)};a^{\left(5\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$. This bilinear concomitant may vanish after implementing the boundary conditions represented by Equations (47) and (54); if it does not vanish, this bilinear concomitant will be reduced to a residual quantity which will comprise only known values of $a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right)$, $u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)$, $\alpha$ and $\delta \alpha$, and will be denoted as ${\left\{{\widehat{P}}^{\left(5\right)}\left(a^{\left(5\right)};u^{\left(5\right)};\alpha ;\delta \alpha \right)\right\}}_{{\alpha }^0}$.

Using the equations underlying the 4^th-LASS together with the relation provided in Equation (51) in Equation (41) yields the following expression for the indirect-effect term ${\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)}\right)}_{{\alpha }^0}\right\}}_{ind}$ in terms of the 5^th-level adjoint functions $a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right)$, for each $j_1=1,\dots ,TP$; $j_2=1,\dots ,j_1$;…;$j_4=1,\dots ,j_3$:

$$\begin{array}{l}{\left\{\delta R^{\left(4\right)}{\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)}\right)}_{{\alpha }^0}\right\}}_{ind}=-{\left\{{\widehat{P}}^{\left(5\right)}\left(a^{\left(5\right)};u^{\left(5\right)};\alpha ;\delta \alpha \right)\right\}}_{{\alpha }^0}\\ +{\left\{{〈a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right),q_V^{\left(5\right)}\left[j_3,j_2,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha ;\delta \alpha \right] 〉}_5\right\}}_{{\alpha }^0}.\end{array}$$

(61)

Adding the result obtained in Equation (61) for the indirect effect term to the result provided in Equation (40) for the direct effect term yields the following expression for the total n^th-order G-variation (of the response) defined in Equation (39):

$$\begin{array}{l}{\left\{\delta R^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha ;\delta u^{\left(5\right)};\delta \alpha \right)\right\}}_{{\alpha }^0}={\left\{\frac{\partial }{\partial \alpha }{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(4\right)}\left(j_4,\dots ,j_1;u^{\left(5\right)};\alpha \right)}}\right\}}_{{\alpha }^0}\delta \alpha , \\ -{\left\{{\widehat{P}}^{\left(5\right)}\left(a^{\left(5\right)};u^{\left(5\right)};\alpha ;\delta \alpha \right)\right\}}_{{\alpha }^0}+{\left\{{〈a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right),q_V^{\left(5\right)}\left[j_3,j_2,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha ;\delta \alpha \right] 〉}_5\right\}}_{{\alpha }^0}.\\ \equiv {\displaystyle \sum _{j_5=1}^{TP}{\left\{R^{\left(5\right)}\left(j_5,\dots ,j_1;u^{\left(5\right)};a^{\left(5\right)};\alpha \right)\right\}}_{\left({\alpha }^0\right)}}\delta {\alpha }_{j_5}.\end{array}$$

(62)

where $R^{\left(5\right)}\left(j_5,\dots ,j_1;u^{\left(5\right)};a^{\left(5\right)};\alpha \right)$ denotes the 5^th-order partial sensitivity of the response $R\left[u^{\left(1\right)}\left(x\right);\alpha \right]$ with respect to the model parameters, evaluated at the nominal parameter values ${\alpha }^0$. The results obtained in this Subsection are summarized in

Table 7. Fifth-Order Sensitivities (n = 5).

5th-Level Forward System	$4^{\mathrm{th}}-\mathrm{Level} \mathrm{Forward} \mathrm{System}+4^{\mathrm{th}}-\mathrm{LASS}: F^{\left(5\right)}{u}^{\left(5\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(5\right)}$; $F^{\left(5\right)}\triangleq diag\left(F^{\left(4\right)},A^{\left(4\right)}\right)$$; \hspace{0.17em}{b}_F^{\left(5\right)}\left(u^{\left(5\right)};\hspace{0.17em}\alpha \right)\triangleq {\left(\hspace{0.17em}{b}_F^{\left(4\right)},b_A^{\left(4\right)}\right)}^{†}={\left[0,0\right]}^{†}$ $u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)={\left[u^{\left(4\right)}\left(j_2,j_1;x\right);\hspace{0.17em}\hspace{0.17em}{a}^{\left(4\right)}\left(j_3,j_2,j_1;x\right);\right]}^{†}$
5th-LVSS	${\left[V^{\left(5\right)}\right]}_{2^5\times 2^5}\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)=q_V^{\left(5\right)}\left[j_3,j_2,j_1;u^{\left(5\right)}\left(j_3{j}_2,j_1;x\right);\alpha ;\delta \alpha \right]$ $\delta u^{\left(5\right)}\left(j_3,j_2,j_1;x\right)={\left(\delta u^{\left(4\right)},\delta a^{\left(4\right)}\right)}^{†}$ $b_V^{\left(5\right)}\hspace{0.17em}\triangleq {\left(b_V^{\left(4\right)},\delta b_A^{\left(4\right)}\right)}^{†}\hspace{0.17em}={\left[0,0\right]}^{†}$$; {\left[V^{\left(5\right)}\right]}_{2^5\times 2^5}\triangleq {\left(\begin{array}{cc}{V}^{\left(4\right)}& {\left[0\right]}_{\left(2^4\times 2^4\right)}\\ V_{21}^{\left(5\right)}& V_{22}^{\left(5\right)}\end{array}\right)}_{2^5\times 2^5}$
5th-Level Hilbert Space; Inner Product	${\mathbf{H}}_5$$; {〈{\eta }^{\left(5\right)}\left(x\right),{\xi }^{\left(5\right)}\left(x\right)〉}_5\triangleq {\displaystyle \sum _{i=1}^{2^5}{〈{\eta }_i^{\left(5\right)}\left(x\right),{\xi }_i^{\left(5\right)}\left(x\right)〉}_0}.$
5th-LASS State Function	$a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right)\triangleq \left[\dots ,a_k^{\left(5\right)}\left(j_4,\dots ,j_1;x\right),\dots \right];\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^5$
5th-LASS	${\left[A^{\left(5\right)}\hspace{0.17em}\right]}_{2^5\times 2^5}\hspace{0.17em}{a}^{\left(5\right)}\left(j_4,\dots ,j_1;x\right)\hspace{0.17em}=s_A^{\left(5\right)}\left[j_4,\dots ,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]\hspace{0.17em}$; ${\left[A^{\left(5\right)}\hspace{0.17em}\right]}_{2^5\times 2^5}\hspace{0.17em}\triangleq {\left[V^{\left(5\right)}\right]}_{2^5\times 2^5}^{*}$$; b_A^{\left(5\right)}\left[a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right);u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]=0$
No. of Distinct Sensitivities	$TP\left(TP+1\right)\left(TP+2\right)\left(TP+3\right)\left(TP+4\right)/5!$$\begin{array}{l}{R}^{\left(5\right)}\left[j_5,\dots ,j_1;u^{\left(5\right)}\left(j_3,j_2,j_1;x\right);a^{\left(5\right)}\left(j_4,\dots ,j_1;x\right);\alpha \right]\\ \triangleq {\partial }^{5}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\dots \partial {\alpha }_{j_5};j_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}{j}_2=1,\dots ,j_1;.....;j_5=1,\dots ,j_4\end{array}$

.

4. Mathematical Modeling of Response-Coupled Linear Forward and Adjoint Systems

The mathematical model of a physical system comprises independent variables (e.g., space, time, etc.), dependent variables (aka “state functions”; e.g., temperature, mass, momentum, etc.) and various parameters (appearing in correlations, coordinates of physical boundaries, etc.), which are all interrelated by equations that usually represent conservation laws. The model parameters usually stem from processes that are external to the system under consideration and are seldom, if ever, known precisely. Without loss of generality, the model parameters can be considered to be real scalar quantities, having known nominal (or mean) values and, possibly, known higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis), which are usually determined from experimental data and/or processes external to the physical system under consideration. These imprecisely known model parameters will be denoted as ${\alpha }_1$,…,${\alpha }_{TP}$, where the subscript “TP” indicates “Total (number of) Parameters”. It will be convenient to consider that these parameters are components of a “vector of parameters” denoted as $\mathsf{\alpha }\triangleq {\left({\alpha }_1,\dots ,{\alpha }_{TP}\right)}^{†}\in {ℝ}^{TP}$, where ${ℝ}^{TP}$ denotes the TP-dimensional subset of the set of real scalars. These model parameters are considered to include imprecisely known geometrical parameters that characterize the physical system’s boundaries in the phase-space of the model’s independent variables. Thus, $\alpha \in {ℝ}^{TP}$ is a TP-dimensional column vector and its components are considered to include all of the uncertain model parameters, including those that may enter into defining the system’s boundary in the phase-space of independent variables. For subsequent developments, it is convenient to denote matrices and vectors using capital and, respectively, lower case bold letters. The symbol “$\triangleq$” will be used to denote “is defined as” or “is by definition equal to”. Transposition will be indicated by a dagger $\left(†\right)$ superscript.

The model is considered to comprise $TI$ independent variables which will be denoted as $x_i,\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI$, and which will be considered as the components of a $TI$-dimensional column vector denoted as $x\triangleq {\left(x_1,\dots ,x_{TI}\right)}^{†}\in {ℝ}^{TI}$, where the sub/superscript “$TI$” denotes the “Total (number of) Independent variables”. The vector $x\in {ℝ}^{TI}$ of independent variables is considered to be defined on a phase-space domain, denoted as $\mathsf{\Omega }\left(\alpha \right)$, $\mathsf{\Omega }\left(\alpha \right)\triangleq \left\{-\infty \le {\lambda }_i\left(\alpha \right)\le x_i\le {\omega }_i\left(\alpha \right)\le \infty ;\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI\right\}$. The lower boundary-point of an independent variable is denoted as ${\lambda }_i\left(\alpha \right)$ (e.g., the inner radius of a sphere or cylinder, the lower range of an energy-variable, etc.), while the corresponding upper boundary-point is denoted as ${\omega }_i\left(\alpha \right)$ (e.g., the outer radius of a sphere or cylinder, the upper range of an energy-variable, etc.). A typical example of boundary conditions that depend on imprecisely known model and geometrical parameters is provided by models based on diffusion theory where the boundaries of the physical domain are facing vacuum. For such models, the boundary conditions for the respective state variables (i.e., particle flux and/or current) are imposed not on the physical boundary but on the “extrapolated boundary” of the respective spatial domain. The “extrapolated boundary” depends both on the imprecisely known physical dimensions of the problem’s domain and also on the medium’s properties, i.e., atomic number densities and microscopic transport cross sections. The boundary of $\mathsf{\Omega }\left(\alpha \right)$, which will be denoted as $\partial \mathsf{\Omega }\left(\alpha \right)$, comprises the set of all of the endpoints ${\lambda }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}{\omega }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI,$ of the respective intervals on which the components of $x$ are defined, i.e., $\partial \mathsf{\Omega }\left(\alpha \right)\triangleq \left\{{\lambda }_i\left(\alpha \right)\cup \hspace{0.17em}{\omega }_i\left(\alpha \right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TI\right\}$.

A linear physical system is generally modeled by a system of coupled linear operator-equations which can be generally represented as follows:

$$L\left(x;\alpha \right)\phi \left(x\right)=q_{\phi }\left(x;\alpha \right)\hspace{0.33em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \mathsf{\Omega }\left(\alpha \right)$$

(63)

where $\phi \left(x\right)\triangleq {\left[{\phi }_1\left(x\right),\dots ,{\phi }_{TD}\left(x\right)\right]}^{†}$ is a $TD$-dimensional column vector of dependent variables, where the sub/superscript “$TD$” denotes the “Total (number of) Dependent variables”. The functions ${\phi }_i\left(x\right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TD$, denote the system’s “dependent variables” (also called “state functions”). The matrix $L\left(x;\alpha \right)\hspace{0.33em}\triangleq \left[L_{ij}\left(x;\alpha \right)\hspace{0.33em}\right],$ $\hspace{0.17em}i,j=1,\dots ,TD$, has dimensions $TD\times TD$. The components $L_{ij}\left(x;\alpha \right)\hspace{0.33em}$ are operators that act linearly on the dependent variables ${\phi }_j\left(x\right)$ and also depend (in general, nonlinearly) on the uncertain parameters $\alpha$. The components of the $TD$-dimensional column vector $q_{\phi }\left(x;\alpha \right)\hspace{0.33em}\triangleq {\left[q_{\phi ,1}\left(x;\alpha \right),\dots .,q_{\phi ,TD}\left(x;\alpha \right)\right]}^{†}$ represent inhomogeneous source terms. The components $q_{\phi ,i}\left(x;\alpha \right)$ of $q_{\phi }\left(x;\alpha \right)$, where the subscript “$\phi$” indicates sources associated with the “forward” system of equations, are also nonlinear functions of $\alpha$. Since the right-side of Equation (63) may contain distributions, the equality in this equation is considered to hold in the weak (i.e., “distributional”) sense. Similarly, all of the equalities that involve differential equations in this work will be considered to hold in the distributional sense.

When $L\left(x;\alpha \right)\hspace{0.33em}$ contains differential operators, a set of boundary and/or initial conditions which define the domain of $L\left(x;\alpha \right)\hspace{0.33em}$ must also be given. Since the complete mathematical model is considered to be linear in $\phi \left(x\right)$, the boundary and/or initial conditions needed to define the domain of $L\left(x;\alpha \right)\hspace{0.33em}$ must also be linear in $\phi \left(x\right)$. Such linear boundary and/or initial conditions are represented in the following operator form:

$$B_{\phi }\left(x;\alpha \right)\hspace{0.33em}\phi \left(x\right)-c_{\phi }\left(x;\alpha \right)=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left(\alpha \right).$$

(64)

In Equation (64), the components $B_{ij}\left(x;\alpha \right);\hspace{0.17em}\hspace{0.17em}i=1,\dots ,N_B;\hspace{0.17em}\hspace{0.17em}j=1,\dots ,TD$, of the $N_B\times TD$-dimensional matrix-operator $B_{\phi }\left(x;\alpha \right)$ are operators that act linearly on $\phi \left(x\right)$ and nonlinearly on $\alpha$; the quantity $N_B$ denotes the total number of boundary and initial conditions. The operator $c_{\phi }\left(x;\alpha \right)\triangleq {\left[c_{\phi ,1}\left(x;\alpha \right),\dots ,c_{\phi ,N_B}\left(x;\alpha \right)\right]}^{†}$ is a $N_B$-dimensional vector comprising components that are operators which in general act nonlinearly on $\alpha$. The subscript “$\phi$” in Equation (64) indicates boundary conditions associated with the forward state function $\phi \left(x\right)$. In this work, capital bold letters will be used to denote matrices while lower case bold letter will be used to denote vectors; the components of these matrices and vectors may be operators rather than just functions.

Physical problems modeled by linear systems and/or operators are naturally defined in Hilbert spaces. The dependent variables ${\phi }_i\left(x\right),\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TD$, for the physical system represented by Equations (63) and (64) are considered to be square-integrable functions of the independent variables and are considered to belong to a Hilbert space which will be denoted as ${\mathrm{H}}_0$, where the subscript “zero” denotes “zeroth-level“ or “original”. Higher-level Hilbert spaces, which will be denoted as ${\mathrm{H}}_1,\hspace{0.17em}\hspace{0.17em}{\mathrm{H}}_2,\hspace{0.17em}$ etc., will also be introduced and used in this work. The Hilbert space ${\mathrm{H}}_0$ is considered to be endowed with the following inner product, denoted as ${〈\phi \left(x\right),\psi \left(x\right)〉}_0$, between two elements $\phi \left(x\right)\in {\mathrm{H}}_0$ and $\psi \left(x\right)\in {\mathrm{H}}_0$:

$$\begin{array}{l}{〈\phi \left(x\right),\psi \left(x\right)〉}_0\triangleq {\displaystyle \underset{{\mathsf{\Omega }}_x}{\int }\phi \left(x\right)\mathbf{·}\psi \left(x\right)d}x\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\phi \left(x\right)\mathbf{·}\psi \left(x\right)\hspace{0.17em}dx}}\\ ={\displaystyle \sum _{i=1}^{TI}{\displaystyle \underset{{\lambda }_1\left(\alpha \right)}{\overset{{\omega }_1\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\dots {\displaystyle \underset{{\lambda }_{TI}\left(\alpha \right)}{\overset{{\omega }_{TI}\left(\alpha \right)}{\int }}{\phi }_i\left(x\right){\psi }_i\left(x\right)\hspace{0.17em}d{x}_1\hspace{0.17em}d{x}_2\dots d{x}_i\dots d{x}_{TI}}}}}.\end{array}$$

(65)

The dot in Equation (65) indicates the “scalar product of two vectors”, which is defined as follows:

$$\phi \left(x\right)\mathbf{·}\psi \left(x\right)\triangleq {\displaystyle \sum _{i=1}^{TD}{\phi }_i\left(x\right){\psi }_i\left(x\right)}$$

(66)

while the product-notation ${\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\left[ \right]\hspace{0.17em}d{x}_i}}$ in Equation (65) denotes the respective multiple integrals. The linear operator $L\left(x;\alpha \right)\hspace{0.33em}$ admits an adjoint operator, denoted as $L^{*}\left(x;\alpha \right)$, which is defined through the following relation for a vector $\psi \left(x\right)\in {\mathrm{H}}_0$:

$${〈\psi \left(x\right), L\left(x;\alpha \right)\hspace{0.33em}\hspace{0.17em}\phi \left(x\right)〉}_0={〈L^{*}\left(x;\alpha \right)\hspace{0.33em}\hspace{0.17em}\psi \left(x\right),\hspace{0.17em}\phi \left(x\right)〉}_0$$

(67)

In Equation (67), the formal adjoint operator $L^{*}\left(x;\alpha \right)\hspace{0.33em}$ is the $TD\times TD$ matrix comprising elements $L_{ji}^{*}\left(x;\alpha \right)\hspace{0.33em}$ obtained by transposing the formal adjoints of the forward operators $L_{ij}\left(x;\alpha \right)$, i.e.,

$$L^{*}\left(x;\alpha \right)\hspace{0.33em}\triangleq \left[L_{ji}^{*}\left(x;\alpha \right)\hspace{0.33em}\right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i,j=1,\dots ,TD\hspace{0.17em},$$

(68)

Hence, the system adjoint to the linear system represented by Equations (63) and (64) can generally be represented as follows:

$$L^{*}\left(x;\alpha \right)\hspace{0.33em}\psi \left(x\right)=q_{\psi }\left(x;\alpha \right)\hspace{0.33em}\hspace{0.33em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x\left(\alpha \right)\hspace{0.17em},$$

(69)

$$B_{\psi }\left(x;\alpha \right)\hspace{0.33em}\psi \left(x\right)\hspace{0.33em}-c_{\psi }\left(x;\alpha \right)\hspace{0.33em}=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left(\alpha \right)\hspace{0.17em}.$$

(70)

When $L\left(x;\alpha \right)$ comprises differential operators, the operations (e.g., integration by parts) that implement the transition from the left-side to the right side of Equation (67) give rise to boundary terms which are collectively called the “bilinear concomitant”. The domain of $L^{*}\left(x;\alpha \right)$ is determined by selecting adjoint boundary and/or initial conditions so as to ensure that the bilinear concomitant vanishes when the selected adjoint boundary conditions are implemented together with the forward boundary conditions given in Equation (64). The adjoint boundary conditions thus selected are represented in operator form by Equation (70), where the subscript “$\psi$” indicates adjoint boundary and/or initial conditions associated with the adjoint state function $\psi \left(x\right)$. In most practical situations, the Hilbert space ${\mathrm{H}}_0$ is self-dual.

The system of equations comprising Equations (63), (64), (69) and (70) will be called the 1^st-Level Forward System (1^st-LFS), since it is the starting point for the computation of the model’s response, and its solution will be needed for the subsequent computation of the first-order sensitivities of the model’s response with respect to the model parameters. The 1^st-LFS can be written compactly in block-matrix form as follows:

$$F^{\left(1\right)}\left(\alpha ;x\right)u^{\left(1\right)}\left(x\right)=\hspace{0.17em}{q}_F^{\left(1\right)}\left(x;\alpha \right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x\hspace{0.17em}\left({\alpha }^0\right)$$

(71)

$$\hspace{0.17em}{b}_F^{\left(1\right)}\left(u^{\left(1\right)};\hspace{0.17em}\alpha \right)=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(72)

where the following definitions were used:

$$F^{\left(1\right)}\left(\alpha ;x\right)\triangleq \left(\begin{array}{cc}L\left(\alpha ;x\right)& 0\\ 0& L^{*}\left(\alpha ;x\right)\end{array}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{u}^{\left(1\right)}\left(x\right)\triangleq \left(\begin{array}{c}\phi \left(x\right)\\ \psi \left(x\right)\end{array}\right);\hspace{0.17em}\hspace{0.17em}{q}_F\left(x;\alpha \right)\triangleq \hspace{0.17em}\left(\begin{array}{c}{q}_{\phi }\left(x;\alpha \right)\\ q_{\psi }\left(x;\alpha \right)\end{array}\right);$$

(73)

$$b_F^{\left(1\right)}\left(u^{\left(1\right)};\hspace{0.17em}\alpha \right)\triangleq \hspace{0.17em}\left(\begin{array}{c}{B}_{\phi }\left(x;\alpha \right)\hspace{0.33em}\phi \left(x\right)-c_{\phi }\left(x;\alpha \right)\\ B_{\psi }\left(x;\alpha \right)\psi \left(x;\alpha \right)\hspace{0.33em}-c_{\psi }\left(x;\alpha \right)\hspace{0.33em}\end{array}\right).$$

(74)

The nominal solution of Equations (63) and (64), which will be denoted as ${\phi }^0\left(x\right)$, and the nominal solution of Equations (69) and (70), which will be denoted as ${\psi }^0\left(x\right)$, are obtained by solving these equations at the nominal (or mean) values of the model parameter ${\alpha }^0$. The superscript “zero” will henceforth be used to denote “nominal” (or, equivalently, “expected” or “mean” values). Thus, the vectors ${\phi }^0\left(x\right)$, ${\psi }^0\left(x\right)$ and ${\alpha }^0$ satisfy the following equations:

$$\begin{array}{l}L\left(x;{\alpha }^0\right){\phi }^0\left(x\right)=q_{\phi }\left(x;{\alpha }^0\right)\hspace{0.33em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x\hspace{0.17em}\left({\alpha }^0\right), B_{\phi }\left(x;{\alpha }^0\right)\hspace{0.33em}{\phi }^0\left(x\right)-c_{\phi }\left(x;{\alpha }^0\right)=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),\\ L^{*}\left(x;{\alpha }^0\right)\hspace{0.33em}{\psi }^0\left(x\right)=q_{\psi }\left(x;{\alpha }^0\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x\left({\alpha }^0\right), B_{\psi }\left(x;{\alpha }^0\right){\psi }^0\left(x\right)\hspace{0.33em}-c_{\psi }\left(x;{\alpha }^0\right)\hspace{0.33em}=0,\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),\end{array}$$

or, equivalently,

$${\left\{F^{\left(1\right)}\left(\alpha ;x\right)u^{\left(1\right)}\left(x\right)\right\}}_{{\alpha }^0}=\hspace{0.17em}{\left\{q_F^{\left(1\right)}\left(x;\alpha \right)\right\}}_{{\alpha }^0},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {\mathsf{\Omega }}_x\hspace{0.17em}\left({\alpha }^0\right)$$

(75)

$${\left\{b_F^{\left(1\right)}\left(u^{\left(1\right)};\hspace{0.17em}\alpha \right)\right\}}_{{\alpha }^0}=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(76)

In Equations (75) and (76), the notation ${\left\{ \right\}}_{{\alpha }^0}$ has been used to indicate that the quantity within the brackets is to be evaluated at the nominal values of the parameters and state functions. This simplified notation is justified by the fact that when the parameters take on their nominal values, it implicitly means that the corresponding state functions also take on their corresponding nominal values. This simplified notation will be used throughout this work.

The relationship shown in Equation (67), which is the basis for defining the adjoint operator, also provides the following fundamental “reciprocity-like” relation between the sources of the forward and the adjoint equations, cf. Equations (63) and (69), respectively:

$${〈\psi \left(x\right),\hspace{0.17em}{q}_{\phi }\left(x;\alpha \right)〉}_0={〈q_{\psi }\left(x;\alpha \right),\hspace{0.17em}\phi \left(x\right)〉}_0$$

(77)

The functional on the right-side of Equation (77) represents a “detector response”, i.e., reaction-rate between the particles and the medium represented by $q_{\psi }\left(x;\alpha \right)$ which is equivalent to the “number of counts” of particles incident on a detector of particles that “measures” the particle flux $\phi \left(x\right)$. Thus, the source term $q_{\psi }\left(x;\alpha \right)\hspace{0.33em}\hspace{0.33em}\triangleq {\left[q_{\psi ,1}\left(x;\alpha \right),\dots .,q_{\psi ,TD}\left(x;\alpha \right)\right]}^{†}$ in Equation (77) is usually associated with the “result of interest” to be measured and/or computed, which is customarily called the system’s “response”. In particular, if $q_{\psi }\left(x;\alpha \right)=\delta \left(x-x_d\right)$, then ${〈q_{\psi }\left(x;\alpha \right),\hspace{0.17em}\phi \left(x\right)〉}_0=\phi \left(x_d\right)$, which means that, in such a case, the right-side of Equation (77) provides the value of the dependent variable (particle flux, temperature, velocity, etc.) at the point in phase-space where the respective measurement is performed.

The relation given in Equation (77) indicates that for linear physical systems, the system’s response may depend not only on the model’s state-functions and on the system parameters, but may also depend on the adjoint function The system’s response will be denoted as $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ and, in the most general case, is a nonlinear operator acting on the model’s forward and adjoint state functions, as well as on imprecisely known parameters, both directly and indirectly through the state functions. The nominal value of the response, $R\left[{\phi }^0\left(x\right),{\psi }^0\left(x\right);{\alpha }^0\right]$, is determined by using the nominal parameter values ${\alpha }^0$, the nominal value ${\phi }^0\left(x\right)$ of the forward state function and the nominal value ${\psi }^0\left(x\right)$ of the adjoint function.

In general, a function-valued (operator) response $R_{op}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ can be represented by a spectral expansion in multidimensional orthogonal polynomials or Fourier series of the form:

$$R_{op}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]={\displaystyle \sum _{m_1}\dots }{\displaystyle \sum _{m_{TI}}{c}_{m_1\dots m_{TI}}{P}_{m_1}\left(x_1\right)P_{m_2}\left(x_2\right)}\hspace{0.17em}\dots P_{m_{TI}}\left(x_{TI}\right),$$

(78)

where the quantities $P_{m_i}\left(x_i\right)$, $i=1,\dots ,TI$, denote the corresponding spectral functions (e.g., orthogonal polynomials or Fourier exponential/trigonometric functions) and where the spectral Fourier coefficients $c_{m_1\dots m_{TI}}$ are defined as follows:

$$c_{m_1\dots m_{TI}}\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}{R}_{op}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]P_{m_1}\left(x_1\right)\dots P_{m_i}\left(x_i\right)\hspace{0.17em}\dots P_{m_{TI}}\left(x_{N_x}\right)d{x}_i}}$$

(79)

The coefficients $c_{m_1\dots m_{TI}}$ can themselves be considered as system responses since the spectral polynomials $P_{m_i}\left(x_i\right)$ are perfectly well known while the expansion coefficients will contain all of the dependencies of the respective response on the imprecisely known model and response parameters. This way, the sensitivity analysis of operator-valued responses can be reduced to the sensitivity analysis of scalar-valued responses.

A measurement of a physical quantity can be represented as a response $R_p\left[\phi \left(x_p\right),\psi \left(x_p\right);\alpha \right]$ located at a specific point, $x_p$, in phase-space, having the following form:

$$R_p\left[\phi \left(x_p\right),\psi \left(x_p\right);\alpha \right]\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]\delta \left(x-x_p\right)d{x}_i}}$$

(80)

where $\delta \left(x-x_p\right)$ denotes the multidimensional Dirac-delta functional. Responses that occur in many fields (e.g., optimization, control, model verification, data assimilation, model validation, model calibration, predictive modeling) are Lagrangians having the following functional form:

$$R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\left\{\phi \left(x\right)q_{\psi }\left(x;\alpha \right)+\psi \left(x\right)\left[L\left(\alpha \right)\phi \left(x\right)-q_{\phi }\left(x;\alpha \right)\right]\right\}\hspace{0.17em}d{x}_i}}$$

(81)

A particular form of the response defined in Equation (81) is the Schwinger “normalization-free Lagrangian”, which takes on the following form [1,2]:

$$R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]\triangleq \frac{\left\{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\phi \left(x\right)q_{\psi }\left(x;\alpha \right)d{x}_i}}\right\}\left\{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\psi \left(x\right)q_{\phi }\left(x;\alpha \right)d{x}_i}}\right\}}{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\psi \left(x\right)L\left(x;\alpha \right)\phi \left(x;x\right)d{x}_i}}}.$$

(82)

The system responses defined in Equation (78) through (82) indicate that their sensitivities (i.e., functional derivatives) with respect to the model parameters can be studied by considering a generic scalar-valued response, which will be denoted as $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$, of the following form:

$$R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}S\hspace{0.17em}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]d{x}_i}},$$

(83)

where $S\hspace{0.17em}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ denotes a suitably Gateaux- (G-) differentiable function of the indicated arguments. In general, $S\hspace{0.17em}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ is nonlinear in $\phi \left(x\right)$, $\psi \left(x\right)$, and $\alpha$, and the components of $\alpha$ are considered to also include parameters that may specifically appear only in the definition of the response under consideration (but which might not appear in the definition of the model). Thus, the (physical) “system” is defined in this work to comprise both the system’s computational model and the system’s response.

The generic response defined in Equation (83) provides the basis for constructing any other responses of specific interest and will therefore be used for the generic “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems”, which will be abbreviated as “n^th-CASAM-L” (where “n” is a finite number that indicates any desired, arbitrarily-high, order) and which will be developed in the remainder of this work. Note that the generic response defined in Equation (83) is, in general, a nonlinear function of all of its arguments, i.e., $S\hspace{0.17em}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ is nonlinear in $\phi \left(x\right)$, $\psi \left(x\right)$, and $\alpha$.

5. The n^th-CASAM-L Methodology for n = 1, 2, 3, 4

5.1. The 1^st-CASAM-L: Summary

The model parameters ${\alpha }_i$ are imprecisely known quantities, so their actual values may differ from their nominal values by quantities denoted as $\delta {\alpha }_i\triangleq {\alpha }_i-{\alpha }_i^0,\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TP.$ Since the model parameters $\alpha$ and the state functions are related to each other through the forward and adjoint systems, it follows that variations $\delta \alpha \triangleq \left(\delta {\alpha }_1,\dots ,\delta {\alpha }_{TP}\right)$ in the model parameters will cause corresponding variations $\delta \phi \triangleq \left(\delta {\phi }_1,\dots ,\delta {\phi }_{TD}\right),$$\hspace{0.17em}\delta {\phi }_i\triangleq {\phi }_i-{\phi }_i^0,$$\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TD$ and $\delta \psi \triangleq \left(\delta {\psi }_1,\dots ,\delta {\psi }_{TD}\right)$, $\hspace{0.17em}\delta {\psi }_i\triangleq {\psi }_i-{\psi }_i^0$, $\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TD$ in the forward and, respectively, adjoint state functions. In turn, the variations $\delta \alpha$, $\delta \phi$, and $\delta \psi$ will cause a response variation $R\left({\phi }^0+\delta \phi ;{\psi }^0+\delta \psi ;{\alpha }^0+\delta \alpha \right)$ around the nominal response value $R\left[{\phi }^0\left(x\right),{\psi }^0\left(x\right);{\alpha }^0\right]$.

The first-order total G-variation $\delta f_i\left[{\eta }_1\left(x\right),\dots ,{\eta }_K\left(x\right);h_1\left(x\right),\dots ,h_K\left(x\right)\right]$ of a function $f\left[{\eta }_1\left(x\right),\dots ,{\eta }_K\left(x\right)\right]$ for arbitrary variations $h\triangleq {\left(h_1,\dots ,h_K\right)}^{†}$ is defined as follows:

$${\left\{\delta f\left[{\eta }_1\left(x\right),\dots ,{\eta }_K\left(x\right);h_1\left(x\right),\dots ,h_K\left(x\right)\right]\right\}}_{{\eta }^0}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\triangleq {\left\{\frac{d}{d\epsilon }f\left[{\eta }_1^0\left(x\right)+\epsilon h_1\left(x\right),\dots ,{\eta }_K^0\left(x\right)+\epsilon h_K\left(x\right)\right]\right\}}_{\epsilon =0}.$$

(84)

The first-order G-variation $\delta f\left(\eta ;h\right)$ is an operator defined on the same domain as $F\left(\eta \right)$, and has the same range as $F\left(\eta \right)$. The G-variation $\delta f\left(\eta ;h\right)$ satisfies the relation $f\left(\eta +\epsilon h\right)-f\left(\eta \right)=\delta f\left(\eta ;h\right)+\Delta \left(h\right)\hspace{0.33em},$ with $\underset{\epsilon \to 0}{\lim }\left[\Delta \left(\epsilon h\right)\right]/\epsilon =0\hspace{0.33em}.$ The existence of the G-variation $\delta f\left(\eta ;h\right)$ does not guarantee its numerical computability. Numerical methods (e.g., Newton’s method and variants thereof) most often require that $\delta f\left(\eta ;h\right)$ be linear in the variations $h$ in a neighborhood $\left({\eta }^0+\epsilon h\right)$ around ${\eta }^0$. The necessary and sufficient conditions for the G-differential $\delta f\left(\eta ;h\right)$ of a nonlinear operator $f\left(\eta \right)$ to be linear in the variations $h$ in a neighborhood $\left(\eta +\epsilon h\right)$ around $\eta$ are as follows:

(i).

$f\left(\eta \right)$ satisfies a weak Lipschitz condition at ${\eta }^0$; i.e.,

$$‖f\left({\eta }^0+\epsilon h\right)-f\left({\eta }^0\right)‖\le k‖\epsilon {\eta }^0‖,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k<\infty$$

(85)

(ii).

For two arbitrary vectors of variations $h_1$ and $h_2$, the operator $f\left(\eta \right)$ satisfies the relation

$$f\left({\eta }^0+\epsilon h_1+\epsilon h_2\right)-f\left({\eta }^0+\epsilon h_1\right)-f\left({\eta }^0+\epsilon h_2\right)+f\left({\eta }^0\right)=o\left(\epsilon \right)$$

(86)

In practice, it is not necessary to investigate if $f\left(\eta \right)$ satisfies the conditions presented in Equations (85) and (86) since it is usually evident if the right-side of Equation (84) is linear (or not) in the variations $h$. A total variation $\delta f\left(\eta ;h\right)$ which is linear in $h$ is called the “total differential of $f\left(u\right)$” and is usually denoted as $Df\left(\eta ;h\right)$. In this case, the partial variation $\delta f\left({\eta }_1,\dots ,{\eta }_K;h_i\right)$ is called the “partial differential $\delta f\left({\eta }_1,\dots ,{\eta }_K;h_i\right)$ of $F\left(\eta \right)$ with respect to ${\eta }_i$”. Thus, the total differential $Df\left(\eta ;h\right)$ is linear in the variations $h_i$ and the following representation holds:

$$Df\left({\eta }_1,\dots ,{\eta }_K;h_1,\dots ,h_K\right)={\displaystyle \sum _{k=1}^K\frac{\partial f\left({\eta }_1,\dots ,{\eta }_K\right)}{\partial {\eta }_i}{h}_i}$$

(87)

where the quantities $\partial f\left({\eta }_1,\dots ,{\eta }_K\right)/\partial {\eta }_i$ denote the partial derivatives of $f\left(\eta \right)$ with respect to its arguments ${\eta }_i$. It will henceforth be assumed that all of the operators considered in this work satisfy the conditions presented in Equations (85) and (86), and therefore admit partial G-derivatives such that the representation shown in Equation (87) exists.

Since this work ultimately aims at deriving the explicit expressions of the n^th-order (i.e., arbitrarily-high order) sensitivities of the response $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ with respect to the model parameters, the proliferation of indices, superscripts and subscripts is unavoidable. To avoid the use of “subscripted-subscripts” as much as possible, the subscripts which will be used for denoting the sensitivities (i.e., partial G-derivatives) of the model response with respect to the model parameters will have the following form: (i) $j_1=1,\dots ,TP$, for first-order sensitivities; (ii) $j_1;j_2=1,\dots ,TP$, for 2^nd-order sensitivities; (iii) $j_1;j_2;j_3=1,\dots ,TP$, for 3^rd-order sensitivities, and so on. Thus, the indices $j_1;j_2;\dots ;j_n=1,\dots ,TP$ will be used for the n^th-order sensitivities.

The 1^st-order Gateaux- (G-) variation, denoted as $\delta R\left({\phi }^0,{\psi }^0,{\alpha }^0;\delta \phi ,\delta \psi ,\delta \alpha \right)$, of the response $R\left(e\right)$ for arbitrary variations $\delta \phi \left(x\right),\delta \psi \left(x\right),\delta \alpha$ in the model parameters and state functions, in a neighborhood $\left[{\phi }^0\left(x\right)+\epsilon \delta \phi \left(x\right),{\psi }^0\left(x\right)+\epsilon \delta \psi \left(x\right);{\alpha }^0+\epsilon \delta \alpha \right]$ around $\left({\phi }^0,{\psi }^0,{\alpha }^0\right)$, where $\epsilon \in \mathrm{F}$ is a real scalar ($\mathrm{F}$ denotes the underlying field of scalars), is defined as follows:

$$\begin{array}{l}{\left\{\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi ,\delta \alpha \right)\right\}}_{{\alpha }^0}\triangleq {\left\{\frac{d}{d\epsilon }R\left[{\phi }^0\left(x\right)+\epsilon \delta \phi \left(x\right),{\psi }^0\left(x\right)+\epsilon \delta \psi \left(x\right);{\alpha }^0+\epsilon \delta \alpha \right]\right\}}_{\epsilon =0}\\ \triangleq {\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \alpha \right)\right]}_{{\alpha }^0}\right\}}_{dir}+{\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}.\end{array}$$

(88)

In Equation (88), the “indirect-effect” term ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \alpha \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ depends only on the variations $\delta \phi$ and $\delta \psi$ in the state functions, and is defined as follows:

$${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}\triangleq {\displaystyle \prod _{i=1}^{TI}{\left\{{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i\left[\frac{\partial S\hspace{0.17em}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]}{\partial \phi }\delta \phi +\frac{\partial S\hspace{0.17em}\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]}{\partial \psi }\delta \psi \right]}\right\}}_{{\alpha }^0}}.$$

(89)

while the “direct-effect” term ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \alpha \right)\right]}_{{\alpha }^0}\right\}}_{dir}$ depends only on the parameter variations $\delta \alpha$ and is defined as follows:

$$\begin{array}{c}{\left\{{\left[\delta R\left(\mathsf{\phi },\mathsf{\psi },\mathsf{\alpha };\delta \mathsf{\alpha }\right)\right]}_{{\mathsf{\alpha }}^0}\right\}}_{dir}\triangleq {\left\{\frac{\partial }{\partial \mathsf{\alpha }}\prod _{i=1}^{TI}\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_{i}S\left[\mathsf{\phi }\left(\mathbf{x}\right),\mathsf{\psi }\left(\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\delta \mathsf{\alpha }\hfill \\ =\prod _{i=1}^{TI}{\left\{\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i\frac{\partial S\left[\mathsf{\phi }\left(\mathbf{x}\right),\mathsf{\psi }\left(\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial \mathsf{\alpha }}\delta \mathsf{\alpha }\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ +\sum _{j=1}^{TI}\prod _{i=1,i\ne j}^{TI}{\left\{\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_{i}S\left[\mathsf{\phi }\left(\dots ,{\omega }_j,\dots ,x_{N_x}\right),\mathsf{\psi }\left(\dots ,{\omega }_j,\dots \right);\mathsf{\alpha }\right]\frac{\partial {\omega }_j\left(\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}\right\}}_{{\mathsf{\alpha }}^0}\delta \mathsf{\alpha }\hfill \\ -\sum _{j=1}^{TI}\prod _{i=1,i\ne j}^{TI}{\left\{\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_{i}S\left[\mathsf{\phi }\left(\dots ,{\lambda }_j,\dots \right),\mathsf{\psi }\left(\dots ,{\lambda }_j,\dots \right);\mathsf{\alpha }\right]\frac{\partial {\lambda }_j\left(\mathsf{\alpha }\right)}{\partial \mathsf{\alpha }}\right\}}_{{\mathsf{\alpha }}^0}\delta \mathsf{\alpha },\hfill \end{array}$$

(90)

where

$$\frac{\partial \left[\hspace{1em}\right]}{\partial \alpha }\delta \alpha \triangleq {\displaystyle \sum _{i=1}^{TP}\frac{\partial \left[\right]}{\partial {\alpha }_i}\delta {\alpha }_i}$$

(91)

In Equations (89) and (90), the notation ${\left\{\right\}}_{{\alpha }^0}$ is used to indicate that the quantity within the brackets is to be evaluated at the nominal values of the parameters and state functions. As has been previously mentioned, this simplified notation is justified by the fact that when the parameters take on their nominal values, it implicitly means that the corresponding state functions also take on their corresponding nominal values. The “direct effect” term ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \alpha \right)\right]}_{{\alpha }^0}\right\}}_{dir}$ defined in Equation (90) depends directly on the parameter variations $\delta \alpha$, and can be computed immediately since it does not depend on the variations $\delta \phi$ and $\delta \psi \left(x\right)$. On the other hand, the “indirect effect” term ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ defined in Equation (89) depends indirectly on the parameter variations $\delta \alpha$, through the variations $\delta \phi \left(x\right)$ and $\delta \psi \left(x\right)$ in the forward state functions $\phi \left(x\right)$ and $\psi \left(x\right)$, and can be computed only after having computed the values of the variations $\delta \phi \left(x\right)$ and $\delta \psi \left(x\right)$.

The variations $\delta \phi \left(x\right)$ and $\delta \psi \left(x\right)$ are the solutions of the equations obtained by the G-differentiating Equations (63), (64), (69) and (70), which yields the following system:, which will be called the 1^st-Level Variational Sensitivity System (1^st-LVSS):

$${\left\{\left(\begin{array}{cc}L \left(\alpha ;x\right)& 0\\ 0& L^{\ast }\left(\alpha ;x\right)\end{array}\right)\left(\begin{array}{c}\delta \phi \left(x\right)\\ \delta \psi \left(x\right)\end{array}\right)\right\}}_{{\alpha }^0}={\left\{\left(\begin{array}{c}{q}_1^{\left(1\right)}\left(\phi ;\alpha ;\delta \alpha \right)\\ q_2^{\left(1\right)}\left(\psi ;\alpha ;\delta \alpha \right)\end{array}\right)\right\}}_{{\alpha }^0}, x\in {\mathsf{\Omega }}_x,$$

(92)

$${\left\{b_V^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta u^{\left(1\right)} ;\delta \alpha \right)\right\}}_{{\alpha }^0}\triangleq {\left\{\left(\begin{array}{c}\delta b_1^{\left(1\right)}\left(\phi ;\alpha ;\delta \phi ;\delta \alpha \right)\\ \delta b_2^{\left(1\right)}\left(\psi ;\alpha ;\delta \psi ;\delta \alpha \right)\end{array}\right)\right\}}_{{\alpha }^0}=\left(\begin{array}{c}0\\ 0\end{array}\right), x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(93)

where

$${\left\{q_1^{(1)}\left(\phi ,\alpha ;\delta \alpha \right) \right\}}_{{\alpha }^0}\triangleq {\left\{\frac{\partial \left[q_{\phi }\left(\alpha \right)-L\left(\alpha \right)\phi \right]}{\partial \alpha } \right\}}_{{\alpha }^0}\delta \alpha ,$$

(94)

$${\left\{q_2^{(1)}\left(\psi ,\alpha ;\delta \alpha \right) \right\}}_{{\alpha }^0}\triangleq {\left\{\frac{\partial \left[q_{\psi }\left(\alpha \right)-L^{\ast }\left(\alpha \right)\psi \left(x\right)\right]}{\partial \alpha }\right\}}_{{\alpha }^0}\delta \alpha ,$$

(95)

The matrices $\partial q_{\phi }\left(\alpha \right)/\partial \alpha$ and $\partial \left[L\left(\alpha \right)\phi \right]/\partial \alpha$, which appear on the right-side of Equation (94), are defined as follows:

$$\frac{\partial q_{\phi }\left(\alpha \right)}{\partial \alpha }\triangleq \left(\begin{array}{ccc}\frac{\partial q_{\phi ,1}}{\partial {\alpha }_1}& \dots & \frac{\partial q_{\phi ,1}}{\partial {\alpha }_{TP}}\\ ⋮& \ddots & ⋮\\ \frac{\partial q_{\phi ,N_{\phi }}}{\partial {\alpha }_1}& \cdots & \frac{\partial q_{\phi ,N_{\phi }}}{\partial {\alpha }_{TP}}\end{array}\right),\frac{\partial \left[L\left(\alpha \right)\phi \right]}{\partial \alpha }\triangleq \left(\begin{array}{ccc}\frac{\partial \left[{\displaystyle \sum _{j=1}^{TD}{L}_{1,j}\left(\alpha \right){\phi }_j}\right]}{\partial {\alpha }_1}& \dots & \frac{\partial \left[{\displaystyle \sum _{j=1}^{TD}{L}_{1,j}\left(\alpha \right){\phi }_j}\right]}{\partial {\alpha }_{TP}}\\ ⋮& \ddots & ⋮\\ \frac{\partial \left[{\displaystyle \sum _{j=1}^{TD}{L}_{N_{\phi },j}\left(\alpha \right){\phi }_j}\right]}{\partial {\alpha }_1}& \cdots & \frac{\partial \left[{\displaystyle \sum _{j=1}^{TD}{L}_{N_{\phi },j}\left(\alpha \right){\phi }_j}\right]}{\partial {\alpha }_{TP}}\end{array}\right).$$

(96)

The matrices $\partial q_{\psi }\left(\alpha \right)/\partial \alpha$ and $\partial \left[L^{\ast }\left(\alpha \right)\psi \right]/\partial \alpha$, which appear on the right-side of Equation (96), have the same structures as the matrices $\partial q_{\phi }\left(\alpha \right)/\partial \alpha$ and $\partial \left[L\left(\alpha \right)\phi \right]/\partial \alpha$, respectively.

Considering the definitions provided in Equations (91), (94) and (95), it follows from Equation (92) that the state functions $\phi \left(x\right)$ and $\psi \left(x\right)$ are implicit functions of the model parameters $\alpha$; therefore, the variations of the variations $\delta \phi \left(x\right)$ and $\delta \psi \left(x\right)$ can be considered to represent the following total differentials:

$$\delta \phi ={\displaystyle \sum _{i=1}^{TP}\frac{\partial \phi }{\partial {\alpha }_i}\delta {\alpha }_i}; \delta \psi ={\displaystyle \sum _{i=1}^{TP}\frac{\partial \psi }{\partial {\alpha }_i}\delta {\alpha }_i}.$$

(97)

Using the results provided in Equations (91), (94)−(97) into Equations (92) and (93), and identifying the expressions that multiply each parameter variation $\delta {\alpha }_i$ yields the following system of equations:

$${\left\{\left(\begin{array}{cc}L\left(\alpha ;x\right)& 0\\ 0& L^{\ast }\left(\alpha ;x\right)\end{array}\right)\left(\begin{array}{c}\partial \phi /\partial {\alpha }_{j_1}\\ \partial \psi /\partial {\alpha }_{j_1}\end{array}\right)\right\}}_{{\alpha }^0}={\left\{\left(\begin{array}{c}{q}_1^{\left(1\right)}/\partial {\alpha }_{j_1}\\ q_2^{\left(1\right)}/\partial {\alpha }_{j_1}\end{array}\right)\right\}}_{{\alpha }^0}; x\in {\mathsf{\Omega }}_x; j_1=1,\dots ,TP,$$

(98)

with the functions $\partial \phi /\partial {\alpha }_{j_1}$ and $\partial \psi /\partial {\alpha }_{j_1}$ subject to the corresponding boundary conditions derived from Equation (93). It is evident from Equation (98) that the determinations of the functions $\partial \phi /\partial {\alpha }_{j_1}$ and $\partial \psi /\partial {\alpha }_{j_1}$ by solving the 1^st-LVSS would require $TP$ large-scale computations, which would be prohibitively expensive in terms of computational resources.

The alternative path for determining the indirect-effect term ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ defined in Equation (89), which avoids the need to compute the functions $\partial \phi /\partial {\alpha }_{j_1}$ and $\partial \psi /\partial {\alpha }_{j_1}$, is provided by the first-order adjoint sensitivity analysis methodology which was conceived by Cacuci [7,8], which will be presented in this subsection in its extended framework, including the computation of response sensitivities to parameters underlying boundary and interface conditions. This extended framework is called the “Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward and Adjoint Linear Systems,” and is abbreviated as 1^st-CASAM-L. The starting point for the 1^st-CASAM-L is the 1^st-LVSS, cf. Equations (92) and (93), which will be written in the following block-matrix form:

$${\left\{V^{\left(1\right)}\left(\alpha ;x\right)\delta u^{\left(1\right)}\left(x\right)\right\}}_{{\alpha }^0}={\left\{q_V^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)\right\}}_{{\alpha }^0} x\in {\mathsf{\Omega }}_x,$$

(99)

$${\left\{b_V^{\left(1\right)}\left[u^{\left(1\right)};\alpha ;\delta u^{\left(1\right)};\delta \alpha \right]\right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(100)

where

$$\begin{array}{l}{\left[V^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}\triangleq F^{\left(1\right)}\left(\alpha ;x\right); \delta u^{\left(1\right)}\left(x\right)\triangleq {\left[\delta \phi \left(x\right),\delta \psi \left(x\right)\right]}^{†}\\ q_V^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)\triangleq {\left[q_1^{\left(1\right)}\left(\phi ;\alpha ;\delta \alpha \right); q_2^{\left(1\right)}\left(\psi ;\alpha ;\delta \alpha \right)\right]}^{†}.\end{array}$$

(101)

The solution $\delta u^{\left(1\right)}\left(x\right)$ of the “1^st-Level Variational Sensitivity System” (1^st-LVSS) will be called the “1^st-level variational sensitivity function, as highlighted by the superscript “(1)”.

The indirect-effect term ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ defined in Equation (89) can be computed by applying of the “First-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward and Adjoint Linear Systems”, abbreviated as 1^st-CASAM-L, which avoids the need for computing the functions $\delta \phi \left(x\right)$ and $\delta \psi \left(x\right)$. The 1^st-CASAM-L uses a Hilbert space, denoted as ${\mathrm{H}}_1$, which comprises square-integrable functions vector-valued elements of the form ${\eta }^{\left(1\right)}\left(x\right)\triangleq {\left[{\eta }_1^{\left(1\right)}\left(x\right),{\eta }_2^{\left(1\right)}\left(x\right)\right]}^{†}$, ${\eta }_i^{\left(1\right)}\left(x\right)\triangleq {\left[{\eta }_{i,1}^{\left(1\right)}\left(x\right),\dots ,{\eta }_{i,j}^{\left(1\right)}\left(x\right),\dots ,{\eta }_{i,TD}^{\left(1\right)}\left(x\right)\right]}^{†}$, $i=1,2$, and which is endowed with an inner product between two elements, ${\eta }^{\left(1\right)}\left(x\right)\in {\mathrm{H}}_1$, ${\xi }^{\left(1\right)}\left(x\right)\in {\mathrm{H}}_1$, denoted as ${〈{\eta }^{\left(1\right)}\left(x\right),{\xi }^{\left(1\right)}\left(x\right)〉}_1$ and defined as follows:

$${〈{\eta }^{\left(1\right)}\left(x\right),{\xi }^{\left(1\right)}\left(x\right)〉}_1\triangleq {\displaystyle \sum _{i=1}^2{〈{\eta }_i^{\left(1\right)}\left(x\right),{\xi }_i^{\left(1\right)}\left(x\right)〉}_0}$$

(102)

In the Hilbert ${\mathrm{H}}_1$, form the inner product of Equation (92) with a yet undefined vector-valued function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}\in {\mathrm{H}}_1$ to obtain the following relation:

$$\begin{array}{l}{\left\{{〈a^{\left(1\right)}\left(x\right),V^{\left(1\right)}\left(x;\alpha \right)\delta u^{\left(1\right)}\left(x\right)〉}_1\right\}}_{{\alpha }^0}={\left\{{〈a^{\left(1\right)}\left(x\right),q_V^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right) 〉}_1\right\}}_{{\alpha }^0}\\ ={\left\{{〈\delta u^{\left(1\right)}\left(x\right),A^{\left(1\right)}\left(x;\alpha \right)a^{\left(1\right)}\left(x\right) 〉}_1\right\}}_{{\alpha }^0}+{\left\{P^{\left(1\right)}\left[\delta u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0},\end{array}$$

(103)

where ${\left\{P^{\left(1\right)}\left[\delta u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$ denotes the bilinear concomitant defined on the phase-space boundary $x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right)$, and where ${\left[A^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}$ is the operator formally adjoint to $V^{\left(1\right)}\left(\alpha ;x\right)$, i.e.,

$${\left[A^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}\triangleq {\left[V^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}^{\ast }=\left(\begin{array}{cc}{L}^{\ast }\left(\alpha ;x\right)& 0\\ 0& L\left(\alpha ;x\right)\end{array}\right)$$

(104)

Require the first term on right-side of the second equality in Equation (103) to represent the indirect-effect term defined in (89), to obtain the following relation:

$${\left[A^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}{a}^{\left(1\right)}\left(x\right) =s_A^{\left(1\right)}\left( u^{\left(1\right)}\left(x\right);\alpha \right),$$

(105)

where the subscript “A” denotes “adjoint” and where the source term $s_A^{\left(1\right)}\left( u^{\left(1\right)}\left(x\right);\alpha \right)$ is defined as follows:

$$s_A^{\left(1\right)}\left[u^{\left(1\right)}\left(x\right);\alpha \right]\triangleq \left(\begin{array}{c}{\left[s_1^{\left(1\right)}\left( u^{\left(1\right)};\alpha \right)\right]}^{†}\\ {\left[s_2^{\left(1\right)}\left( u^{\left(1\right)};\alpha \right)\right]}^{†}\end{array}\right)\triangleq \left(\begin{array}{c}{\left[\partial S\left( u^{\left(1\right)};\alpha \right)/\partial \phi \right]}^{†}\\ {\left[\partial S\left( u^{\left(1\right)};\alpha \right)/\partial \psi \right]}^{†}\end{array}\right).$$

(106)

Implement the boundary conditions given in Equation (93) into Equation (103), thereby eliminating the remaining unknown boundary-values of the functions $\delta \phi$ and $\delta \psi$ from the expression of the bilinear concomitant ${\left\{P^{\left(1\right)}\left[\delta u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]\right\}}_{{\alpha }^0}$ by selecting appropriate boundary conditions for the function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$, to ensure that Equation (105) is well-posed while being independent of unknown values of $\delta \phi$, $\delta \psi$, and $\delta \alpha$. The boundary conditions thus chosen for the function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$ can be represented in operator form as follows:

$${\left\{b_A^{\left(1\right)}\left[ u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\right]\right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right).$$

(107)

The selection of the boundary conditions for the adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$ represented by (107) eliminates the appearance of the unknown values of $\delta u^{\left(1\right)}\left(x\right)$ in ${\left\{P^{\left(1\right)}\left[\delta u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$ and reduces this bilinear concomitant to a residual quantity that contains boundary terms involving only known values of $\phi \left(x\right)$, $\psi \left(x\right)$, $a^{\left(1\right)}\left(x\right)$, $\alpha$ and $\delta \alpha$. This residual quantity will be denoted as ${\left\{{\widehat{P}}^{\left(1\right)}\left[ u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$. In general, this residual quantity does not automatically vanish, although it may do so occasionally.

The system of equations comprising Equation (105) together with the boundary conditions represented by Equation (107) constitutes the 1^st-Level Adjoint Sensitivity System (1^st-LASS). The solution $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$ of the 1^st-LASS is called the 1^st-level adjoint function. The 1^st-LASS is called “first-level” (as opposed to “first-order”) because it does not contain any differential or functional-derivatives, but its solution, $a^{\left(1\right)}\left(x\right)$, will be used below to compute the first-order sensitivities of the response with respect to the model parameters. This terminology will be also used in the sequel, when deriving the expressions for the higher-order sensitivities.

It follows from Equations (103) and (105) that the indirect-effect term ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ can be expressed in terms of the 1^st-level adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$ as follows:

$$\begin{array}{l}{\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}={\left\{{〈a^{\left(1\right)}\left(x\right),q_V^{\left(1\right)}\left( u^{\left(1\right)};\alpha ;\delta \alpha \right) 〉}_1\right\}}_{{\alpha }^0} -{\left\{{\widehat{P}}^{\left(1\right)}\left[u^{\left(1\right)};a^{\left(1\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}\\ ={\left\{{\displaystyle \sum _{k=1}^2{〈a_k^{\left(1\right)}\left(x\right),q_k^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)〉}_0}\right\}}_{{\alpha }^0}-{\left\{{\widehat{P}}^{\left(1\right)}\left[u^{\left(1\right)};a^{\left(1\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}\equiv {\left\{\delta R\left(u^{\left(1\right)};a^{\left(1\right)};\alpha ;\delta \alpha \right)\right\}}_{ind}.\end{array}$$

(108)

As indicated by the last equality in Equation (108), the variations $\delta \phi$ and $\delta \psi$ have been eliminated from the original expression of the indirect-effect term, which now instead depends on the 1^st-level adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$. As indicated in Equation (105), solving the 1^st-LASS entails the following operations: (i) inverting (i.e., solving) the original left-side of the adjoint equation with the source ${\left[\partial S\left(u^{\left(1\right)};\alpha \right)/\partial \phi \right]}^{†}$ to obtain the 1st-level adjoint function $a_1^{\left(1\right)}\left(x\right)$; and (ii) inverting the original left-side of the forward equation with the source ${\left[\partial S\left(u^{\left(1\right)};\alpha \right)/\partial \psi \right]}^{†}$ to obtain the 1st-level adjoint function $a_2^{\left(1\right)}\left(x\right)$. It is very important to note that the 1st-LASS is independent of parameter variations $\delta \alpha$. Hence, the 1^st-LASS needs to be solved only once (as opposed to the 1st-LVSS, which would need to be solved anew for each parameter variation) to determine the 1^st-level adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$. Subsequently, the “indirect-effect term” ${\left\{{\left[\delta R\left(\phi ,\psi ,\alpha ;\delta \phi ,\delta \psi \right)\right]}_{{\alpha }^0}\right\}}_{ind}$ is computed efficiently and exactly by simply performing the integrations over the adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$, as indicated on the right-side of Equation (108).

The total 1^st-order sensitivity of the response $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ to the model parameters is obtained by adding the expressions of the direct-effect term defined in Equation (90) and indirect-effect term as obtained in Equation (108), which yields the following expression:

$$\begin{array}{l}{\left\{\delta R\left[u^{\left(1\right)}\left(x\right),\alpha ;\delta \phi ,\delta \psi ,\delta \alpha \right]\right\}}_{{\alpha }^0}={\displaystyle \prod _{i=1}^{TI}{\left\{{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i\frac{\partial S \left[\phi \left(x\right),\psi \left(x\right);\alpha \right]}{\partial \alpha }\delta \alpha }\right\}}_{{\alpha }^0}}\\ +{\displaystyle \sum _{j=1}^{TI}{\displaystyle \prod _{i=1,i\ne j}^{TI}{\left\{{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_{i}S \left[\phi \left(\dots ,{\omega }_j,\dots ,x_{N_x}\right),\psi \left(\dots ,{\omega }_j,\dots \right);\alpha \right]\frac{\partial {\omega }_j\left(\alpha \right)}{\partial \alpha }}\right\}}_{{\alpha }^0}\delta \alpha }}\\ -{\displaystyle \sum _{j=1}^{TI}{\displaystyle \prod _{i=1,i\ne j}^{TI}{\left\{{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i}S \left[\phi \left(\dots ,{\lambda }_j,\dots \right),\psi \left(\dots ,{\lambda }_j,\dots \right);\alpha \right]\frac{\partial {\lambda }_j\left(\alpha \right)}{\partial \alpha }\right\}}_{{\alpha }^0}\delta \alpha }},\\ +{\left\{{\displaystyle \sum _{k=1}^2{〈a_k^{\left(1\right)}\left(x\right),q_k^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)〉}_0}\right\}}_{{\alpha }^0}-{\left\{{\widehat{P}}^{\left(1\right)}\left[u^{\left(1\right)};a^{\left(1\right)};\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}\\ \equiv {\left\{\delta R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}.\end{array}$$

(109)

As indicated in the last equality in Equation (109), the expression of the first-order G-variation $\delta R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]$ no longer depends on the variations $\delta \phi$ and $\delta \psi$, which are expensive to compute, but instead depends on the 1^st-level adjoint function $a^{\left(1\right)}\left(x\right)\triangleq {\left[a_1^{\left(1\right)}\left(x\right),a_2^{\left(1\right)}\left(x\right)\right]}^{†}$, which is as inexpensive to compute as the original state function(s). In particular, the expression in Equation (109) also reveals that the sensitivities of the response $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ to parameters that characterize the system’s boundary and/or internal interfaces can arise both from the direct-effect and indirect-effect terms. It also follows from Equation (109) that the total 1^st-order G-variation $\delta R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]$ can be expressed in terms of the 1^st-level adjoint functions as follows:

$${\left\{\delta R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}={\displaystyle \sum _{j_1=1}^{TP}{\left\{\frac{\partial R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]}{\partial {\alpha }_{j_1}}\right\}}_{{\alpha }^0}\delta {\alpha }_{j_1}}$$

(110)

where the quantities ${\left\{\partial R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]/\partial {\alpha }_{j_1}\right\}}_{{\alpha }^0}$ represent the 1^st-order response sensitivities with respect to the model parameters, evaluated at the nominal values of the state functions and model parameters. For the subsequent computation of higher-order responses sensitivities, it is convenient to represent the 1^st-order response sensitivities as follows:

$$\begin{array}{l}\frac{\partial R\left[u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]}{\partial {\alpha }_{j_1}}\triangleq R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]\\ \triangleq {\displaystyle \prod _{k=1}^{TI}{\displaystyle \underset{{\lambda }_k\left(\alpha \right)}{\overset{{\omega }_k\left(\alpha \right)}{\int }}d{x}_k{S}^{\left(1\right)} \left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]}} , j_1=1,\dots ,TP.\end{array}$$

(111)

In particular, if the residual bilinear concomitant is non-zero, the functions $S^{\left(1\right)} \left[j1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]$ would contain suitably defined Dirac delta-functionals for expressing the respective non-zero boundary terms as volume-integrals over the phase-space of the independent variables. Dirac-delta functionals would also be needed to represent, within $S^{\left(1\right)} \left[j1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]$, the terms containing the derivatives of the boundary end-points with respect to the model and/or response parameters. As indicated by Equation (111), the computation of the 1^st-order response sensitivities to all model parameters requires two large-scale computations (which is just as many as solving the original system) namely: one large-scale computation for computing the first-level adjoint state function $a_1^{\left(1\right)}\left(x\right)$ and one large-scale computation for computing the first-level adjoint state function $a_2^{\left(1\right)}\left(x\right)$.

5.2. The 2^nd-CASAM-L: Summary

For each value of the index $j_1=1,\dots ,TP$, the first-order sensitivity $R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]$ will be considered to play the role of a “model response.” The starting point for the development of the 2^nd-CASAM-L is provided by the expressions of the sensitivities provided in terms of the 1^st-level adjoint functions, namely Equation (111), which indicates that the computation of the first-order sensitivities $R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]$ using the 1^st-CASAM-L requires the determination of the functions $u^{\left(1\right)}\left(x\right)$ and $a^{\left(1\right)}\left(x\right)$. In view of the developments underlying the 1^st-CASAM-L, the functions $u^{\left(1\right)}\left(x\right)$ and $a^{\left(1\right)}\left(x\right)$ are the solutions of the following system of equations, written in block-matrix form:

$${\left\{F^{\left(2\right)}\left(\alpha ;x\right)u^{\left(2\right)}\left(x\right)\right\}}_{{\alpha }^0}= {\left\{q_F^{\left(2\right)}\left(u^{\left(1\right)};x;\alpha \right)\right\}}_{{\alpha }^0}, x\in {\mathsf{\Omega }}_x \left({\alpha }^0\right)$$

(112)

$${\left\{b_F^{\left(2\right)}\left(u^{\left(2\right)}; \alpha \right)\right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(113)

where

$$\begin{array}{l}{F}^{\left(2\right)}\left(\alpha ;x\right)\triangleq \left(\begin{array}{cc}{\left[F^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}& 0\\ 0& {\left[A^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}\end{array}\right); u^{\left(2\right)}\left(x\right)\triangleq \left(\begin{array}{c}{u}^{\left(1\right)}\left(x\right)\\ a^{\left(1\right)}\left(x\right)\end{array}\right);\\ q_F^{\left(2\right)}\left(u^{\left(1\right)};x;\alpha \right)\triangleq \left(\begin{array}{c}{q}_F^{\left(1\right)}\left(x;\alpha \right)\\ s^{\left(1\right)}\left( u^{\left(1\right)}\left(x\right);\alpha \right)\end{array}\right); b_F^{\left(2\right)}\left(u^{\left(2\right)}; \alpha \right)\triangleq \left(\begin{array}{c}{b}_F^{\left(1\right)}\left(x;\alpha \right)\\ b_A^{\left(1\right)}\left[ u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha ;\right]\end{array}\right)\end{array}$$

(114)

The system of equations represented by Equations (112) and (113) will be called the 2^nd-Level Forward System, (2^nd-LFS) since it plays the same role as the First-Level (original) Forward System (1^st-LFS) has played for the computation of the first-order response sensitivities. In terms of the function $u^{\left(2\right)}\left(x\right)$, each 1^st-order sensitivity can be compactly written, for each $j_1=1,\dots ,TP$, as follows:

$$\partial R\left(\phi ;\psi ;\alpha \right)/\partial {\alpha }_{j_1}\equiv R^{\left(1\right)}\left[j_1;u^{\left(2\right)}\left(x\right);\alpha \right]\equiv {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(1\right)}\left[j_1;u^{\left(2\right)}\left(x\right);\alpha \right]}}.$$

(115)

There are $TP\left(TP+1\right)/2!$ distinct 2^nd-order sensitivities of the response with respect to the model and response parameters. The 2^nd-order total sensitivity of the model response is obtained by applying the definition of the 1^st-order G-differential to Equation (111), which yields the following result, for each $j_1=1,\dots ,TP$:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j_1; u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(2\right)}\left(x\right);\delta \alpha \right]\right\}}_{{\alpha }^0}\triangleq \left\{\frac{d}{d\epsilon }\right.\left[{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left({\alpha }^0+\epsilon \delta \alpha \right)}{\overset{{\omega }_i\left({\alpha }^0+\epsilon \delta \alpha \right)}{\int }}d{x}_i{S}^{\left(1\right)}\left(j_1;\right.u^{\left(2\right)}+\epsilon \delta u^{\left(2\right)};}}\right.\\ {\left.{\left.\left.\alpha +\epsilon \delta \alpha \right)\right]}_{{\alpha }^0}\right\}}_{\epsilon =0}={\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta \alpha \right]}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(2\right)}\left(x\right)\right]}_{{\alpha }^0}\right\}}_{ind},\end{array}$$

(116)

where the direct-effect term ${\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta \alpha \right]}_{{\alpha }^0}\right\}}_{dir}$ is defined as follows:

$${\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta \alpha \right]}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{\frac{\partial }{\partial \alpha }{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)}\left(x\right);\alpha \right]}}\right\}}_{{\alpha }^0}\delta \alpha ,$$

(117)

while the indirect-effect term ${\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(2\right)}\left(x\right)\right]}_{{\alpha }^0}\right\}}_{ind}$ is defined as follows:

$${\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(2\right)}\left(x\right)\right]}_{{\alpha }^0}\right\}}_{ind}\triangleq {{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}\left[\frac{\partial S^{\left(1\right)}\left(j_1;u^{\left(1\right)};a^{\left(1\right)};\alpha \right)}{\partial u^{\left(2\right)}}\delta u^{\left(2\right)}\left(x\right)\right]}}}_{{\alpha }^0},$$

(118)

where

$$\frac{\partial \left[ \right]}{\partial u^{\left(2\right)}}\delta u^{\left(2\right)}\left(x\right)\triangleq \frac{\partial \left[ \right]}{\partial \phi }\delta \phi \left(x\right)+\frac{\partial \left[ \right]}{\partial \psi }\delta \psi \left(x\right)+\frac{\partial \left[ \right]}{\partial a_1^{\left(1\right)}}\delta a_1^{\left(1\right)}\left(x\right)+\frac{\partial \left[ \right]}{\partial a_2^{\left(1\right)}}\delta a_2^{\left(1\right)}\left(x\right).$$

(119)

As indicated by the subscript “zero,” the expression in Equation (118) is to be computed at nominal parameter and state function values.

The direct-effect term ${\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta \alpha \right]}_{{\alpha }^0}\right\}}_{dir}$ can be computed immediately. The indirect-effect term, ${\left\{\delta R^{\left(1\right)}{\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(2\right)}\left(x\right)\right]}_{{\alpha }^0}\right\}}_{ind}$, however, depends on the variation $\delta u^{\left(2\right)}\left(x\right)$. In turn, the variation $\delta u^{\left(2\right)}\left(x\right)$ is the solution of G-differentiated 2^nd-LFS or, equivalently, the solution of the system obtained by concatenating the 1^st-LVSS with the equations obtained by G-differentiating the 1^st-LASS, cf. Equations (105) and (107). Either way, one obtains the following system of equations:

$${\left\{{\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}\delta u^{\left(2\right)}\left(x\right)\right\}}_{{\alpha }^0}={\left\{q_V^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\right\}}_{{\alpha }^0}, x\in {\mathsf{\Omega }}_x,$$

(120)

$${\left\{b_V^{\left(2\right)}\left[u^{\left(2\right)};\alpha ;\delta u^{\left(2\right)}\left(x\right);\delta \alpha \right]\right\}}_{{\alpha }^0}\triangleq {\left\{ \left(\begin{array}{c} b_V^{\left(1\right)}\left(u^{\left(1\right)};\alpha ;\delta u^{\left(1\right)};\delta \alpha \right)\\ \delta b_A^{\left(1\right)}\left(u^{\left(2\right)};\alpha ;\delta u^{\left(2\right)};\delta \alpha \right)\end{array}\right)\right\}}_{{\alpha }^0}=\left(\begin{array}{c}0\\ 0\end{array}\right), x {\mathsf{\Omega }}_x\left({\alpha }^0\right),$$

(121)

where:

$$\begin{array}{l}{\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}\triangleq \left(\begin{array}{cc}{V}^{\left(1\right)}& {\left[0\right]}_{2\times 2}\\ V_{21}^{\left(2\right)}& V_{22}^{\left(2\right)}\end{array}\right)=\left(\begin{array}{cccc}L\left(\alpha \right)& 0& 0& 0\\ 0& L^{*}\left(\alpha \right)& 0& 0\\ -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \phi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \psi }& L^{*}\left(\alpha \right)& 0\\ -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \phi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \psi }& 0& L\left(\alpha \right)\end{array}\right)\\ V_{21}^{\left(2\right)}\triangleq \left(\begin{array}{cc}-\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \phi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \psi }\\ -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \phi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \psi }\end{array}\right); V^{\left(1\right)}={\left[F^{\left(1\right)}\left(\alpha ;x\right)\right]}_{2\times 2}; V_{22}^{\left(2\right)}={\left[A^{\left(1\right)}\left(\alpha \right)\right]}_{2\times 2};\end{array}$$

(122)

$$\delta u^{\left(2\right)}\left(x\right)\triangleq \left(\begin{array}{c}\delta u^{\left(1\right)}\left(x\right)\\ \delta a^{\left(1\right)}\left(x\right)\end{array}\right)=\left(\begin{array}{c}\delta \phi \left(x\right)\\ \delta \psi \left(x\right)\\ \delta a_1^{\left(1\right)}\left(x\right)\\ \delta a_2^{\left(1\right)}\left(x\right)\end{array}\right); q_V^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\triangleq \left(\begin{array}{c}{q}_1^{\left(2\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)\\ q_2^{\left(2\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)\\ q_3^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\\ q_4^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\end{array}\right)\triangleq \left(\begin{array}{c} q_1^{\left(1\right)}\left(\phi ;\alpha ;\delta \alpha \right)\\ q_2^{\left(1\right)}\left(\psi ;\alpha ;\delta \alpha \right)\\ p_1^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\\ p_2^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\end{array}\right).$$

(123)

$$p_1^{\left(2\right)}\left(\phi ;\psi ;a_1^{\left(1\right)};\alpha ;\delta \alpha \right)\triangleq \frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \alpha \partial \phi }\delta \alpha -\frac{\partial \left[L^{*}\left(\alpha \right)a_1^{\left(1\right)}\right]}{\partial \alpha }\delta \alpha ,$$

(124)

$$p_2^{\left(2\right)}\left(u^{\left(1\right)};a_2^{\left(1\right)};\alpha ;\delta \alpha \right)\triangleq \frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \alpha \partial \psi }\delta \alpha -\frac{\partial \left[L\left(\alpha \right)a_2^{\left(1\right)}\right]}{\partial \alpha }\delta \alpha ,$$

(125)

$$\begin{array}{l}{\left\{\delta b_A^{\left(1\right)}\left(u^{\left(1\right)};a^{\left(1\right)};\alpha ;\delta u^{\left(1\right)};\delta a_1^{\left(1\right)};\delta a_2^{\left(1\right)};\delta \alpha \right)\right\}}_{{\alpha }^0}\triangleq {\left\{\frac{\partial b_A^{\left(1\right)}\left(u^{\left(1\right)};a^{\left(1\right)};\alpha \right)}{\partial \phi }\delta \phi +\frac{\partial b_A^{\left(1\right)}\left(u^{\left(1\right)};a^{\left(1\right)};\alpha \right)}{\partial \psi }\delta \psi \right\}}_{{\alpha }^0}\\ +{\left\{\frac{\partial b_A^{\left(1\right)}\left(u^{\left(1\right)};a^{\left(1\right)};\alpha \right)}{\partial a_1^{\left(1\right)}}\delta a_1^{\left(1\right)}+\frac{\partial b_A^{\left(1\right)}\left(u^{\left(1\right)};a^{\left(1\right)};\alpha \right)}{\partial a_2^{\left(1\right)}}\delta a_2^{\left(1\right)}\right\}}_{{\alpha }^0}+{\left\{\frac{\partial b_A^{\left(1\right)}\left(u^{\left(1\right)};a^{\left(1\right)};\alpha \right)}{\partial \alpha }\delta \alpha \right\}}_{{\alpha }^0}=0, x\in \partial {\mathsf{\Omega }}_x\left({\alpha }^0\right).\end{array}$$

(126)

The system comprising Equations (120) and (121) will be called the “2^nd-Level Variational Sensitivity System” (2^nd-LVSS), and its solution, $\delta u^{\left(2\right)}\left(x\right)$, will be called the 2^nd-level variational function.

All of the components of the matrices and vectors defined in Equations (120) and (121) are to be computed at nominal parameter and state function values. The matrix $V_{21}^{\left(2\right)}$ depends only the system’s response and is responsible for coupling the forward and adjoint systems. Although the forward and adjoint systems are coupled, they could nevertheless be solved successively rather than simultaneously, because the matrix ${\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}$ is a 4 × 4 block-diagonal matrix.

In principle, the 2^nd-LVSS could be solved for each possible component $\delta {\alpha }_{j_2}$, $j_2=1,\dots ,j_1$, of the vector of parameter variations $\delta \alpha$ to obtain the 2^nd-level variational function $\delta u^{\left(2\right)}\left(x\right)$. Subsequently, $\delta u^{\left(2\right)}\left(x\right)$ could be used together with the known parameter variations $\delta \alpha$ in Equation (118) to compute the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(1\right)}\left(x\right);\delta a^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}$. Computing ${\left\{\delta R^{\left(1\right)}\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(1\right)}\left(x\right);\delta a^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}$ would require at least $2TP\left(TP+1\right)$ large-scale computations for solving the 2^nd-LVSS for every independent component (${\alpha }_{j_2}$, ${\alpha }_{j_1}$, $j_1=1,\dots ,TP;$ $j_2=1,\dots ,j_1$) of the vectors of parameter variations $\delta \alpha$.

Following the principles introduced by Cacuci (2018), the application of the 2^nd-CASAM-L avoids the need for solving the 2^nd-LVSS, by expressing the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left(j_1\right)\right\}}_{ind}$ defined in Equation (118) in an alternative manner, in terms of the solution of a 2^nd-Level Adjoint Sensitivity System (2^nd-LASS), which is constructed so as to eliminate the appearance of the 2^nd-level variational vector $\delta u^{\left(2\right)}\left(x\right)$ in the alternative expression of the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left(j_1\right)\right\}}_{ind}$. The construction of the requisite 2^nd-LASS commences by introducing a Hilbert space, denoted as ${\mathrm{H}}_2$, comprising square-integrable vector-valued elements of the form ${\eta }^{\left(2\right)}\left(x\right)\triangleq {\left[{\eta }_1^{\left(2\right)}\left(x\right),{\eta }_2^{\left(2\right)}\left(x\right),{\eta }_3^{\left(2\right)}\left(x\right),{\eta }_4^{\left(2\right)}\left(x\right)\right]}^{†}\in {\mathrm{H}}_2$, with components ${\eta }_i^{\left(2\right)}\left(x\right)\triangleq {\left[{\eta }_{i,1}^{\left(2\right)}\left(x\right),\dots ,{\eta }_{i,j}^{\left(2\right)}\left(x\right),\dots ,{\eta }_{i,TD}^{\left(2\right)}\left(x\right)\right]}^{†}, i=1,2,3,4,$. The inner product between two elements, ${\eta }^{\left(2\right)}\left(x\right)\in {\mathrm{H}}_2$ and ${\xi }^{\left(2\right)}\left(x\right)\in {\mathrm{H}}_2$, in this Hilbert space will be denoted as ${〈{\eta }^{\left(2\right)}\left(x\right),{\xi }^{\left(2\right)}\left(x\right)〉}_2$ and is defined as follows:

$${〈{\eta }^{\left(2\right)}\left(x\right),{\xi }^{\left(2\right)}\left(x\right)〉}_2\triangleq {\displaystyle \sum _{i=1}^2{〈{\eta }_i^{\left(2\right)}\left(x\right),{\xi }_i^{\left(2\right)}\left(x\right)〉}_2}={\displaystyle \sum _{k=1}^{2^2}{〈{\eta }_k^{\left(2\right)}\left(x\right),{\xi }_k^{\left(2\right)}\left(x\right)〉}_0}$$

(127)

In the Hilbert ${\mathrm{H}}_2$, form the inner product of Equation (120) with a yet undefined function $a^{\left(2\right)}\left(j_1;x\right)\triangleq {\left[a_1^{\left(2\right)}\left(j_1;x\right),a_2^{\left(2\right)}\left(j_1;x\right),a_3^{\left(2\right)}\left(j_1;x\right),a_4^{\left(2\right)}\left(j_1;x\right)\right]}^{†}\in {\mathrm{H}}_2$,$j_1=1,\dots ,TP$, to obtain the following relation:

$$\begin{array}{c}{\left\{{〈a^{\left(2\right)}\left(j_1;x\right),V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\delta u^{\left(2\right)}\left(x\right)〉}_2\right\}}_{{\alpha }^0}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left\{{〈a^{\left(2\right)}\left(j_1;x\right),q^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\hspace{0.17em}〉}_2\right\}}_{{\alpha }^0}\\ ={\left\{{〈\delta u^{\left(2\right)}\left(x\right),A^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)a^{\left(2\right)}\left(j_1;x\right)\hspace{0.17em}〉}_2\right\}}_{{\alpha }^0}+{\left\{P^{\left(2\right)}\left[\delta u^{\left(2\right)}\left(x\right);u^{\left(1\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0},\end{array}$$

(128)

where ${\left\{P^{\left(2\right)}\left[\delta u^{\left(2\right)}\left(x\right);u^{\left(1\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$ denotes the bilinear concomitant defined on the phase-space boundary $x\in \partial {{\Omega }}_x\left({\alpha }^0\right)$ and where ${\left[A^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}\triangleq {\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}^{*}$ is the operator formally adjoint to ${\left[V^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{{}_{2^2\times 2^2}}$, having the following form:

$$\begin{array}{c}{\left[A^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}={\left[\begin{array}{cc}{A}^{\left(1\right)}\left(\alpha \right)& {\left[V_{21}^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}^{*}\\ {\left[0\right]}_{\left(2\times 2\right)}& V^{\left(1\right)}\left(\alpha \right)\end{array}\right]}_{2^2\times 2^2}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\left(\begin{array}{cccc}{L}^{*}\left(\alpha \right)& 0& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \phi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \phi }\\ 0& L\left(\alpha \right)& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \psi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \psi }\\ 0& 0& L\left(\alpha \right)& 0\\ 0& 0& 0& L^{*}\left(\alpha \right)\end{array}\right)}_{2^2\times 2^2}.\end{array}$$

(129)

The first term on right-side of the second equality in Equation (128) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left(j_1\right)\right\}}_{ind}$ defined in Equation (118). This requirement is satisfied requiring that the 2^nd-level adjoint function $a^{\left(2\right)}\left(j_1;x\right)\triangleq {\left[a_1^{\left(2\right)}\left(j_{1}1;x\right),\dots ,a_k^{\left(2\right)}\left(j_1;x\right),\dots ,a_4^{\left(2\right)}\left(j_1;x\right)\right]}^{†};\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^2$,$j_1=1,\dots ,TP$ be the solution of the following 2^nd-Level Adjoint Sensitivity System (2^nd-LASS):

$${\left\{{\left[A^{\left(2\right)}\left(u^{\left(1\right)};\alpha \right)\right]}_{2^2\times 2^2}{a}^{\left(2\right)}\left(j_1;x\right)\right\}}_{{\alpha }^0}\hspace{0.17em}={\left\{s_A^{\left(2\right)}\left(j_1;u^{\left(2\right)};\alpha \right)\right\}}_{{\alpha }^0}\hspace{0.17em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TP,$$

(130)

$${\left\{b_A^{\left(2\right)}\left[u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {{\Omega }}_x\left({\alpha }^0\right),\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TP.$$

(131)

where the following definitions were used for each $j_1=1,\dots ,TP$:

$$\begin{array}{c}{s}_A^{\left(2\right)}\left(j_1;u^{\left(2\right)};\alpha \right)\triangleq {\left[s_1^{\left(2\right)}\left(j_1;u^{\left(2\right)};x\right),s_2^{\left(2\right)}\left(j_1;u^{\left(2\right)};x\right),s_3^{\left(2\right)}\left(j_1;u^{\left(2\right)};x\right),s_4^{\left(2\right)}\left(j_1;u^{\left(2\right)};x\right)\right]}^{†}\\ \triangleq \hspace{0.17em}\left[\frac{\partial S^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi },\frac{\partial S^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \psi },\right.{\left.\frac{\partial S^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_1^{\left(1\right)}},\frac{\partial S^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_2^{\left(1\right)}}\right]}^{†}.\end{array}$$

(132)

The adjoint boundary conditions represented by Equation (131) are selected so as to eliminate, in conjunction with the boundary conditions provided in Equation (121), all of the unknown values of the functions $\delta u^{\left(2\right)}\left(x\right)$ in the expression of the bilinear concomitant ${\left\{P^{\left(2\right)}\left[u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha ;\delta u^{\left(2\right)}\left(x\right);\delta \alpha \right]\right\}}_{{\alpha }^0}$. The bilinear concomitant may vanish after these boundary conditions are implemented, but if it does not, it will be reduced to a residual quantity which will be denoted as ${\widehat{P}}^{\left(2\right)}\left[u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]$ and which will comprise only known values of $u^{\left(2\right)}\left(x\right)$, $a^{\left(2\right)}\left(j_1;x\right)$, $\alpha$ and $\delta \alpha$.

The results provided in Equation (130), (131) and (128) are employed in Equation (118) to obtain the following expression for the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(1\right)}\left(x\right);\delta a^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}$ in terms of the 2^nd-level adjoint functions $a^{\left(2\right)}\left(j_1;x\right)$:

$$\begin{array}{l}For\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,TP:\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\left\{\delta R^{\left(1\right)}\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(1\right)}\left(x\right);\delta a^{\left(1\right)}\left(x\right)\right]\right\}}_{ind}\\ =\hspace{0.17em}{\left\{{〈a^{\left(2\right)}\left(j_1;x\right),q^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\hspace{0.17em}〉}_2\right\}}_{{\alpha }^0}-{\left\{{\widehat{P}}^{\left(2\right)}\left[u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}.\end{array}$$

(133)

Inserting the expressions that define the vector $q^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)$ from Equation (122) into Equation (133) and adding the resulting expression for the indirect-effect term with the expression of the direct-effect term given in Equation (117) yields the following expression for the total second-order G-differential of the response $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$:

$$\begin{array}{c}{\left\{\delta R^{\left(1\right)}\left[j_1;u^{\left(2\right)}\left(x\right);\alpha ;\delta u^{\left(2\right)}\left(x\right);\delta \alpha \right]\right\}}_{{\alpha }^0}={\left\{\frac{\partial }{\partial \alpha }{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}}\right\}}_{{\alpha }^0}\delta \alpha \\ +\hspace{0.17em}{\left\{{\displaystyle \sum _{k=1}^{2^2}{〈a_k^{\left(2\right)}\left(j_1;x\right),q_k^{\left(1\right)}\left(\phi ;\alpha ;\delta \alpha \right)\hspace{0.17em}〉}_0}\right\}}_{{\alpha }^0}\equiv {\displaystyle \sum _{j_2=1}^{TP}{\left\{R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}\delta {\alpha }_{j_2}}\hspace{0.17em},\end{array}$$

(134)

where $R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]$ denotes the 2^nd-order partial sensitivity of the response with respect to the model parameters, evaluated at the nominal parameter values ${\alpha }^0$, and has the following expression for $j_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_2=1,\dots ,j_1$:

$$\begin{gathered} R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]\equiv \frac{\partial R^{\left(1\right)}\left[j_1;u^{\left(1\right)}\left(x\right);a^{\left(1\right)};\alpha \right]}{\partial {\alpha }_{j_2}}\equiv \frac{{\partial }^{2}R\left[u^{\left(1\right)}\left(x\right);\alpha \right]}{\partial {\alpha }_{j_2}\partial {\alpha }_{j_1}} \\ =\frac{\partial }{\partial {\alpha }_{j_2}}\hspace{0.17em}{\left\{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}}\right\}}_{{\alpha }^0}+{\left\{{〈a_1^{\left(2\right)}\left(j_1;x\right),\frac{\partial \left[q_{\phi }\left(\alpha \right)-L\left(\alpha \right)\phi \left(x\right)\right]}{\partial {\alpha }_{j_2}}\hspace{0.17em}〉}_0\right\}}_{{\alpha }^0} \\ +\hspace{0.17em}\hspace{0.17em}{\left\{{〈a_2^{\left(2\right)}\left(j_1;x\right),\frac{\partial \left[q_{\psi }\left(\alpha \right)-L^{*}\left(\alpha \right)\psi \left(x\right)\right]}{\partial {\alpha }_{j_2}}〉}_0\right\}}_{{\alpha }^0}\hspace{0.17em}+\hspace{0.17em}{\left\{{〈a_3^{\left(2\right)}\left(j_1;x\right),\frac{{\partial }^{2}S\left(u^{\left(1\right)}\left(x\right);\alpha \right)}{\partial {\alpha }_{j_2}\partial \phi }-\frac{\partial \left[L^{*}\left(\alpha \right)a_1^{\left(1\right)}\right]}{\partial {\alpha }_{j_2}}\hspace{0.17em}〉}_0\right\}}_{{\alpha }^0} \\ +\hspace{0.17em}\hspace{0.17em}{\left\{{〈a_4^{\left(2\right)}\left(j_1;x\right),\frac{{\partial }^{2}S\left(u^{\left(1\right)}\left(x\right);\alpha \right)}{\partial {\alpha }_{j_2}\partial \psi }-\frac{\partial \left[L\left(\alpha \right)a_2^{\left(1\right)}\right]}{\partial {\alpha }_{j_2}}\hspace{0.17em}〉}_0\right\}}_{{\alpha }^0}\hspace{0.17em}-\hspace{0.17em}{\left\{\frac{\partial }{\partial {\alpha }_{j_2}}{\widehat{P}}^{\left(2\right)}\left[u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}. \end{gathered}$$

(135)

Since the 2^nd-LASS is independent of parameter variations $\delta \alpha$, the exact computation of all of the partial second-order sensitivities $R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]$ requires at most $TP$ large-scale (adjoint) computations using the 2^nd-LASS, rather than $O\left(T{P}^2\right)$ large-scale computations as would be required by forward methods. It is important to note that by solving the 2^nd-LASS $TP$-times, the “off-diagonal” 2^nd-order mixed sensitivities ${\partial }^{2}R/\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}$ will be computed twice, in two different ways (i.e., using distinct 2^nd-level adjoint functions), thereby providing an independent intrinsic (numerical) verification that the 1^st- and 2^nd-order response sensitivities are computed accurately. The information provided by the 1^st-order sensitivities usually indicates which 2^nd-order sensitivities are important and which could be neglected. Therefore, it is useful to prioritize the computation of the 2^nd-order sensitivities by using the rankings of the relative magnitudes of the 1st-order sensitivities as a “priority indicator”: the larger the magnitude of the relative 1^st-order sensitivity, the higher the priority for computing the corresponding 2^nd-order sensitivities. Also, since vanishing 1^st-order sensitivities may indicate critical points of the response in the phase-space of model parameters, it is also of interest to compute the 2^nd-order sensitivities that correspond to vanishing 1^st-order sensitivities. In practice, only those 2^nd-order partial sensitivities which are deemed important would need to be computed.

Dirac delta-functionals in Equation (135) may need to be used for expressing the non-zero residual terms in the respective residual bilinear concomitant and/or the terms containing derivatives with respect to the lower- and upper-boundary points, so that the expression of the partial second-order sensitivities $R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]$ can be written in the following form, in preparation for computing the 3^rd-order sensitivities:

$$R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]={\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]}}.$$

(136)

5.3. The 3^rd-CASAM-L: Summary

For each value of the indices $j_1=1,\dots ,TP$,$j_2=1,\dots ,j_1$, the 2^nd-order sensitivity $R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]$ will be considered to play the role of a “model response.” The starting point for the development of the 3^rd-CASAM-L is provided by the expressions of the sensitivities provided in terms of the 1^st-level adjoint functions, namely Equation (136), which indicates that the computation of the 2^nd-order sensitivities $R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]$ using the 2^nd-CASAM-L requires the determination of the functions $u^{\left(2\right)}\left(x\right)$ and $a^{\left(2\right)}\left(j_1;x\right)$. Recalling the developments of the 2^nd-CSASM-L presented in Section 5.2, the functions $u^{\left(2\right)}\left(x\right)$ and $a^{\left(2\right)}\left(j_1;x\right)$ are the solutions of the following system of equations, written in block-matrix form:

$${\left\{F^{\left(3\right)}\left(\alpha ;x\right)u^{\left(3\right)}\left(j_1;x\right)\right\}}_{{\alpha }^0}=\hspace{0.17em}{\left\{q_F^{\left(3\right)}\left(u^{\left(2\right)};x;\alpha \right)\right\}}_{{\alpha }^0},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in {{\Omega }}_x\hspace{0.17em}\left({\alpha }^0\right)$$

(137)

$$\hspace{0.17em}{\left\{b_F^{\left(3\right)}\left[u^{\left(3\right)}\left(j_1\right);\hspace{0.17em}\alpha \right]\right\}}_{{\alpha }^0}=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}x\in \partial {{\Omega }}_x\left({\alpha }^0\right),$$

(138)

where:

$$\begin{gathered} F^{\left(3\right)}\left(\alpha ;x\right)\triangleq \left(\begin{array}{cc}{\left[F^{\left(2\right)}\left(\alpha ;x\right)\right]}_{2\times 2}& 0\\ 0& {\left[A^{\left(2\right)}\left(\alpha ;x\right)\right]}_{2\times 2}\end{array}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{u}^{\left(3\right)}\left(j_1;x\right)\triangleq \left(\begin{array}{c}{u}^{\left(2\right)}\left(x\right)\\ a^{\left(2\right)}\left(j_1;x\right)\end{array}\right); \\ {q}_F^{\left(3\right)}\left(u^{\left(2\right)};x;\alpha \right)\triangleq \left(\begin{array}{c}{q}_F^{\left(2\right)}\left(x;\alpha \right)\\ s^{\left(2\right)}\left(\hspace{0.17em}{u}^{\left(2\right)}\left(x\right);\alpha \right)\end{array}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{b}_F^{\left(3\right)}\left[u^{\left(3\right)}\left(j_1;x\right);\hspace{0.17em}\alpha \right]\triangleq \left(\begin{array}{c}\hspace{0.17em}{b}_F^{\left(2\right)}\left(u^{\left(2\right)};\hspace{0.17em}\alpha \right)\\ b_A^{\left(2\right)}\left[\hspace{0.17em}{u}^{\left(3\right)}\left(j_1;x\right);\alpha ;\right]\end{array}\right). \end{gathered}$$

(139)

The system of equations represented by Equations (137) and (138) can be called the 3^rd-Level Forward System. In terms of the function $u^{\left(3\right)}\left(j_1;x\right)$, each 2^nd-order sensitivity can be compactly written as follows:

$$\begin{array}{l}{\partial }^{2}R\left(\phi ;\psi ;\alpha \right)/\partial {\alpha }_{j_1}\partial {\alpha }_{j_2}\equiv R^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]\\ \equiv R^{\left(2\right)}\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha \right]={\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i{S}^{\left(2\right)}\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha \right]}}\hspace{0.17em}.\end{array}$$

(140)

There are $TP\left(TP+1\right)\left(TP+2\right)/3!$ distinct 3^rd-order sensitivities of the response with respect to the model and response parameters. The third-order sensitivities of the response $R\left[\phi \left(x\right),\psi \left(x\right);\alpha \right]$ with respect to the model parameters are obtained by determining the first-order G-differential of the 2^nd-order sensitivities $R^{\left(2\right)}\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha \right]$, which are defined in Equations (136) or (140). By definition, the total G-differential of $R^{\left(2\right)}\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha \right]$ is obtained as follows:

$$\begin{gathered} {\left\{\delta R^{\left(2\right)}\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}\triangleq {\left\{\frac{d}{d\epsilon }{\left[\left[{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left({\alpha }^0+\epsilon \delta \alpha \right)}{\overset{{\omega }_i\left({\alpha }^0+\epsilon \delta \alpha \right)}{\int }}d{x}_i{S}^{\left(2\right)}\left(j_2,j_1;u^{\left(3\right)}+\epsilon \delta u^{\left(3\right)};\alpha +\epsilon \delta \alpha \hspace{0.17em}\right)}}\right.\right]}_{{\alpha }^0}\right\}}_{\epsilon =0} \\ ={\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]}_{{\alpha }^0}\right\}}_{dir}+{\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta u^{\left(3\right)}\right]}_{{\alpha }^0}\right\}}_{ind}, \end{gathered}$$

(141)

where the direct-effect term ${\left\{\delta R^{\left(2\right)}\left[j_2,j_1;\dots ;\delta \alpha \right]\right\}}_{dir}$ depends directly on the parameter variations and is defined as follows:

$${\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]}_{{\alpha }^0}\right\}}_{dir}\triangleq {\left\{\frac{\partial }{\partial \alpha }{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_k{S}^{\left(2\right)}\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha \right]}}\right\}}_{{\alpha }^0}\delta \alpha ,\hspace{0.17em}$$

(142)

while the “indirect-effect term” ${\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta u^{\left(3\right)}\right]}_{{\alpha }^0}\right\}}_{ind}$ depends indirectly on the parameter variations through the variations in the forward and adjoint state functions and is defined as follows:

$${\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta u^{\left(3\right)}\right]}_{{\alpha }^0}\right\}}_{ind}\triangleq \hspace{0.17em}{\left\{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\alpha \right)}{\overset{{\omega }_i\left(\alpha \right)}{\int }}d{x}_i\frac{\partial S^{\left(2\right)}\left[j_2,j_1;u^{\left(2\right)}\left(x\right);a^{\left(2\right)}\left(j_1;x\right);\alpha \right]}{\partial u^{\left(3\right)}\left(j_1;x\right)}}}\right\}}_{{\alpha }^0}\delta u^{\left(3\right)},$$

(143)

where

$$\begin{array}{l}\frac{\partial \left[ \right]}{\partial u^{\left(3\right)}\left(j1;x\right)}\delta u^{\left(3\right)}\left(j_1;x\right)\\ \equiv \frac{\partial \left[ \right]}{\partial \phi }\delta \phi \left(x\right)+\frac{\partial \left[ \right]}{\partial \psi }\delta \psi \left(x\right)+{\displaystyle \sum _{k=1}^2\frac{\partial \left[ \right]}{\partial a_k^{\left(1\right)}}}\delta a_k^{\left(1\right)}\left(x\right)+{\displaystyle \sum _{k=1}^{2^2}\frac{\partial \left[ \right]}{\partial a_k^{\left(2\right)}\left(j_1\right)}\delta a_k^{\left(2\right)}\left(j_1;x\right),}\end{array}$$

(144)

The indirect-effect term defined in Equation (143) can be computed only after having determined the vector of variations $\hspace{0.17em}\delta u^{\left(3\right)}\left(j_1;x\right)\triangleq {\left[\delta u^{\left(2\right)}\left(x\right);\delta a^{\left(2\right)}\left(j_1;x\right)\right]}^{†}$, which is the solution of the G-differentiated 3^rd-LFS. The variation $\delta a^{\left(2\right)}\left(j_1;x\right)\triangleq {\left[\delta a_1^{\left(2\right)}\left(j_1;x\right),\delta a_2^{\left(2\right)}\left(j_1;x\right),\delta a_3^{\left(2\right)}\left(j_1;x\right),\delta a_4^{\left(2\right)}\left(j_1;x\right)\right]}^{†}$ is the solution of the G-differentiated 2^nd-LASS, while $\delta u^{\left(2\right)}\left(x\right)$ is the solution of the 2^nd-LVSS. Thus, the G-differentiated 3^rd-LFS is obtained by concatenating the 2^nd-LVSS with the G-differentiated 2^nd-LASS (including the corresponding boundary/initial conditions), which yields the following system of 8 coupled equations, which can be represented in the block-matrix form, for $j_1=1,\dots ,TP$, as follows:

$$V^{\left(3\right)}\left[j_1;u^{\left(2\right)}\left(x\right)\right]\hspace{0.17em}\delta u^{\left(3\right)}\left(j_1;x\right)\hspace{0.17em}=q_V^{\left(3\right)}\left[j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right],\hspace{0.17em}\hspace{0.17em}x\in {{\Omega }}_x,$$

(145)

$$b_V^{\left(3\right)}\left[j_1;u^{\left(3\right)};\alpha ;\delta u^{\left(3\right)};\delta \alpha \right]\triangleq \left(\begin{array}{c}{b}_V^{\left(2\right)}\left[u^{\left(2\right)};\alpha ;\delta u^{\left(2\right)};\delta \alpha \right]\\ \delta b_A^{\left(2\right)}\left[j_1;u^{\left(3\right)};\alpha ;\delta u^{\left(3\right)};\delta \alpha \right]\end{array}\right)\hspace{0.17em}=\left(\begin{array}{c}0\\ 0\end{array}\right),\hspace{0.17em}\hspace{0.17em}x\in \partial {{\Omega }}_x\left({\alpha }^0\right).$$

(146)

The system represented by Equations (145) and (146) is called the 3^rd-Level Variational Sensitivity System (3^rd-LVSS) and the solution, $\delta u^{\left(3\right)}\left(x\right)$, of the 3^rd-LVSS will be called the “3^rd-level variational sensitivity function.” The components of the matrices and vectors which appear in the 3^rd-LVSS are to be computed at nominal parameter and state function values, although the corresponding indication has been omitted in order to simplify the notation. The matrices and vectors which appear in the 3^rd-LVSS, Equations (145) and (146), are defined as follows:

$$\begin{array}{l}{q}_V^{\left(3\right)}\left[u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]\triangleq \left(\begin{array}{c}{q}_V^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\\ p^{\left(2\right)}\left[j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta \alpha \right]\end{array}\right)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\triangleq {\left[q_1^{\left(3\right)},q_2^{\left(3\right)},q_3^{\left(3\right)},q_4^{\left(3\right)},q_5^{\left(3\right)}\left(j1\right),q_6^{\left(3\right)}\left(j_1\right),q_7^{\left(3\right)}\left(j_1\right),q_8^{\left(3\right)}\left(j_1\right)\right]}^{†};\end{array}$$

(147)

$$\begin{array}{l}{p}_1^{\left(2\right)}\left[j_1;u^{\left(3\right)}\left(j_1;x\right)\hspace{0.17em};\alpha ;\delta \alpha \right]\triangleq -\frac{\partial \left[L^{*}\left(\alpha \right)a_1^{\left(2\right)}\left(j_1;x\right)\right]}{\partial \alpha }\delta \alpha +\left[\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \alpha \partial \phi \partial \phi }\delta \alpha \right]a_3^{\left(2\right)}\left(j_1;x\right)\hspace{0.17em}\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\left[\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \alpha \partial \psi \partial \phi }\delta \alpha \right]a_4^{\left(2\right)}\left(j_1;x\right)\hspace{0.17em}+\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;\hspace{0.17em}{u}^{\left(2\right)};\alpha \right)}{\partial \alpha \partial \phi }\delta \alpha ,\end{array}$$

(148)

$$\begin{array}{l}{p}_2^{\left(2\right)}\left[j_1;u^{\left(3\right)}\left(j_1;x\right)\hspace{0.17em};\alpha ;\delta \alpha \right]\triangleq -\frac{\partial \left[L\left(\alpha \right)a_2^{\left(2\right)}\left(j_1;x\right)\right]}{\partial \alpha }\delta \alpha +\left[\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \alpha \partial \phi \partial \psi }\delta \alpha \right]a_3^{\left(2\right)}\left(j_1;x\right)\hspace{0.17em}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\left[\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \alpha \partial \psi \partial \psi }\delta \alpha \right]a_4^{\left(2\right)}\left(j_1;x\right)+\frac{\partial S^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \alpha \partial \psi }\delta \alpha ,\end{array}$$

(149)

$$p_3^{\left(2\right)}\left[j_1;u^{\left(3\right)}\left(j_1;x\right)\hspace{0.17em};\alpha ;\delta \alpha \right]\triangleq -\frac{\partial \left[L\left(\alpha \right)a_3^{\left(2\right)}\left(j_1;x\right)\right]}{\partial \alpha }\delta \alpha +\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \alpha \partial a_1^{\left(1\right)}}\delta \alpha ,$$

(150)

$$p_4^{\left(2\right)}\left[j_1;u^{\left(3\right)}\left(j_1;x\right)\hspace{0.17em};\alpha ;\delta \alpha \right]\triangleq -\frac{\partial \left[L^{*}\left(\alpha \right)a_4^{\left(2\right)}\left(j_1;x\right)\right]}{\partial \alpha }\delta \alpha +\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \alpha \partial a_2^{\left(1\right)}}\delta \alpha \hspace{0.17em},$$

(151)

$$\begin{gathered} {\left[V^{\left(3\right)}\left(j_1;u^{\left(2\right)}\right)\right]}_{2^3\times 2^3}\hspace{0.17em}\triangleq {\left(\begin{array}{cc}{V}^{\left(2\right)}\left(u^{\left(1\right)}\right)& {\left[0\right]}_{4\times 4}\\ V_{21}^{\left(3\right)}\left(j_1;u^{\left(2\right)}\right)& V_{22}^{\left(3\right)}\left(u^{\left(1\right)}\right)\end{array}\right)}_{2^3\times 2^3};\hspace{0.17em} \\ {\left[V_{22}^{\left(3\right)}\left(u^{\left(1\right)}\right)\right]}_{4\times 4}\triangleq \left(\begin{array}{cccc}{L}^{*}\left(\alpha \right)& 0& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \phi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \phi }\\ 0& L\left(\alpha \right)& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \psi }& -\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \psi }\\ 0& 0& L\left(\alpha \right)& 0\\ 0& 0& 0& L^{*}\left(\alpha \right)\end{array}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em} \end{gathered}$$

(152)

The components $v_{21}^{\left(3\right)}\left(i,j\right)$ of the matrix $V_{21}^{\left(3\right)}\left(j1\right)\triangleq {\left[v_{21}^{\left(3\right)}\left(i,j\right)\right]}_{4\times 4};\hspace{0.17em}\hspace{0.17em}i,j=1,\dots ,4$ are defined as follows:

$$v_{21}^{\left(3\right)}\left(1,1\right)=-\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \phi \partial \phi }{a}_3^{\left(2\right)}\left(j_1;x\right)\hspace{0.17em}-\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \phi \partial \phi }{a}_4^{\left(2\right)}\left(j_1;x\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi \partial \phi },$$

(153)

$$v_{21}^{\left(3\right)}\left(1,2\right)=-\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \phi \partial \psi }{a}_3^{\left(2\right)}\left(j_1;x\right)\hspace{0.17em}-\frac{{\partial }^{2}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \psi \partial \phi }{a}_4^{\left(2\right)}\left(j_1;x\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi \partial \psi },$$

(154)

$$v_{21}^{\left(3\right)}\left(1,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi \partial a_1^{\left(1\right)}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}_{21}^{\left(3\right)}\left(1,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi \partial a_2^{\left(1\right)}},$$

(155)

$$v_{21}^{\left(3\right)}\left(2,1\right)=-\frac{{\partial }^{3}S\left(u^{\left(2\right)};\alpha \right)}{\partial \phi \partial \phi \partial \psi }{a}_3^{\left(2\right)}\left(j_1;x\right)-\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \psi \partial \psi }{a}_4^{\left(2\right)}\left(j_1;x\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi \partial \psi },$$

(156)

$$v_{21}^{\left(3\right)}\left(2,2\right)=-\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \phi \partial \psi \partial \psi }{a}_3^{\left(2\right)}\left(j_1;x\right)-\frac{{\partial }^{3}S\left(u^{\left(1\right)};\alpha \right)}{\partial \psi \partial \psi \partial \psi }{a}_4^{\left(2\right)}\left(j_1;x\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \psi \partial \psi },$$

(157)

$$v_{21}^{\left(3\right)}\left(2,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_1^{\left(1\right)}\partial \psi },\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}_{21}^{\left(3\right)}\left(2,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_2^{\left(1\right)}\partial \psi },$$

(158)

$$v_{21}^{\left(3\right)}\left(3,1\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi \partial a_1^{\left(1\right)}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}_{21}^{\left(3\right)}\left(3,2\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \psi \partial a_1^{\left(1\right)}},$$

(159)

$$v_{21}^{\left(3\right)}\left(3,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_1^{\left(1\right)}\partial a_1^{\left(1\right)}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}_{21}^{\left(3\right)}\left(3,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_2^{\left(1\right)}\partial a_1^{\left(1\right)}},$$

(160)

$$v_{21}^{\left(3\right)}\left(4,1\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \phi \partial a_2^{\left(1\right)}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}_{21}^{\left(3\right)}\left(4,2\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial \psi \partial a_2^{\left(1\right)}},$$

(161)

$$v_{21}^{\left(3\right)}\left(4,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_1^{\left(1\right)}\partial a_2^{\left(1\right)}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}_{21}^{\left(3\right)}\left(4,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j_1;u^{\left(2\right)};\alpha \right)}{\partial a_2^{\left(1\right)}\partial a_2^{\left(1\right)}}.$$

(162)

The need for solving the 3^rd-LVSS can be avoided by expressing the indirect-effect term ${\left\{\delta R^{\left(2\right)}{\left[j_2;j_1;u^{\left(3\right)}\left(j_1;x\right);\alpha ;\delta u^{\left(3\right)}\right]}_{{\alpha }^0}\right\}}_{ind}$ defined in Equation (143) in terms of the solutions of a 3^rd-Level Adjoint Sensitivity System (3^rd-LASS), which does not involve any variations in the state functions and which is constructed by implementing the same sequence of logical steps as used to construct the 1^st-LASS and the 2^nd-LASS. The 3^rd-LASS is constructed in a Hilbert space, denoted as ${\mathrm{H}}_3$, which comprises vector-valued elements of the form ${\eta }^{\left(3\right)}\left(x\right)\triangleq {\left[{\eta }_1^{\left(3\right)}\left(x\right),\dots ,{\eta }_8^{\left(3\right)}\left(x\right)\right]}^{†}\in {\mathrm{H}}_3$, with ${\eta }_i^{\left(3\right)}\left(x\right)\triangleq {\left[{\eta }_{i,1}^{\left(3\right)}\left(x\right),\dots ,{\eta }_{i,j}^{\left(3\right)}\left(x\right),\dots ,{\eta }_{i,TD}^{\left(3\right)}\left(x\right)\right]}^{†},\hspace{0.17em}\hspace{0.17em}i=1,‥,8$. The inner product between two elements, ${\eta }^{\left(3\right)}\left(x\right)\in {\mathrm{H}}_3$ and ${\xi }^{\left(3\right)}\left(x\right)\in \hspace{0.17em}{\mathrm{H}}_3$, is denoted as ${〈{\eta }^{\left(3\right)}\left(x\right),{\xi }^{\left(3\right)}\left(x\right)〉}_3$ and is defined as follows:

$${〈{\eta }^{\left(3\right)}\left(x\right),{\xi }^{\left(3\right)}\left(x\right)〉}_3\triangleq {\displaystyle \sum _{i=1}^8{〈{\eta }_i^{\left(3\right)}\left(x\right),{\xi }_i^{\left(3\right)}\left(x\right)〉}_0}.$$

(163)

In the Hilbert ${\mathrm{H}}_3$, form the inner product of Equation (145) with a yet undefined vector-function $a^{\left(3\right)}\left(j_2,j_1;x\right)\triangleq {\left[a_1^{\left(3\right)}\left(j_2,j_1;x\right),\dots ,a_8^{\left(3\right)}\left(j_2,j_1;x\right)\right]}^{†}\in {\mathrm{H}}_3$,$j_1=1,\dots ,TP;\hspace{0.17em}\hspace{0.17em}{j}_1=1,\dots ,j_2$, to obtain the following relation:

$$\begin{array}{c}{\left\{{〈{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right),{\mathbf{V}}^{\left(3\right)}\left[j_1;{\mathbf{u}}^{\left(2\right)}\left(\mathbf{x}\right)\right]\delta {\mathbf{u}}^{\left(3\right)}\left(\mathbf{x}\right)〉}_3\right\}}_{{\mathsf{\alpha }}^0}={\left\{{〈{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right),{\mathbf{q}}_V^{\left(3\right)}\left[j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]〉}_3\right\}}_{{\mathsf{\alpha }}^0}.\hfill \\ ={\left\{{〈\delta {\mathbf{u}}^{\left(3\right)}\left(\mathbf{x}\right),{\mathbf{A}}^{\left(3\right)}\left(j_1\right){\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)〉}_3\right\}}_{{\mathsf{\alpha }}^0}+{\left\{P^{\left(3\right)}\left[\delta {\mathbf{u}}^{\left(3\right)}\left(\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0},\hfill \end{array}$$

(164)

where ${\left\{P^{\left(3\right)}\left[\delta {\mathbf{u}}^{\left(3\right)}\left(\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ denotes the bilinear concomitant defined on the phase-space boundary $\mathbf{x}\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right)$ and where ${\left[{\mathbf{A}}^{\left(3\right)}\left(j_1;{\mathbf{u}}^{\left(2\right)}\right)\right]}_{2^3\times 2^3}\triangleq {\left[{\mathbf{V}}^{\left(3\right)}\left(j_1;{\mathbf{u}}^{\left(2\right)}\right)\right]}_{2^3\times 2^3}^{*}$ has the following form:

$${\left[{\mathbf{A}}^{\left(3\right)}\left(j_1;{\mathbf{u}}^{\left(2\right)}\right)\right]}_{2^3\times 2^3}={\left(\begin{array}{cc}{\mathbf{A}}^{\left(2\right)}& {\left[{\mathbf{V}}_{21}^{\left(3\right)}\left(j_1;{\mathbf{u}}^{\left(2\right)}\right)\right]}^{*}\\ {\left[\mathbf{0}\right]}_{4\times 4}& {\left[{\mathbf{V}}_{22}^{\left(3\right)}\right]}^{*}\end{array}\right)}_{2^3\times 2^3}.$$

(165)

The first term on right-side of the second equality in Equation (164) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;{\mathbf{u}}^{\left(2\right)};{\mathbf{a}}^{\left(2\right)}\left(j_1\right);\mathsf{\alpha };\delta {\mathbf{u}}^{\left(2\right)};\delta {\mathbf{a}}^{\left(2\right)}\left(j_1\right)\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}$ defined in Equation (142). This requirement is satisfied by requiring that the 3^rd-level adjoint function ${\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\triangleq {\left[{\mathbf{a}}_1^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right),\dots ,{\mathbf{a}}_8^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\right]}^{†}\triangleq {\left[\dots ,{\mathbf{a}}_k^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right),\dots \right]}^{†};k=1,\dots ,2^3;$ $j_1=1,\dots ,TP;$ $j_2=1,\dots ,j_1$, be the solution of the following 3^rd-Level Adjoint Sensitivity System (3^rd-LASS):

$${\left\{{\mathbf{A}}^{\left(3\right)}\left(j_1\right){\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\right\}}_{{\mathsf{\alpha }}^0} ={\left\{{\mathbf{s}}_A^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0} ,$$

(166)

$${\left\{{\mathbf{b}}_A^{\left(3\right)}\left[{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}=\mathbf{0}, \mathbf{x}\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right).$$

(167)

The block-vector ${\mathbf{s}}_A^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq {\left[{\mathbf{s}}_1^{\left(3\right)}\left(j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right),\dots ,{\mathbf{s}}_8^{\left(3\right)}\left(j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right)\right]}^{†}$ on the right-side of Equation (166) comprises, for each $j_1=1,\dots ,TP ; j_2=1,\dots ,j_1$, eight components (which are themselves block-vectors) defined as follows:

$${\mathbf{s}}_1^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial \mathsf{\phi }},$$

(168)

$${\mathbf{s}}_2^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial \mathsf{\psi }},$$

(169)

$${\mathbf{s}}_3^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial {\mathbf{a}}_1^{\left(1\right)}},$$

(170)

$${\mathbf{s}}_4^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial {\mathbf{a}}_2^{\left(1\right)}},$$

(171)

$${\mathbf{s}}_5^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial {\mathbf{a}}_1^{\left(2\right)}\left(j_1;\mathbf{x}\right)},$$

(172)

$${\mathbf{s}}_6^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial {\mathbf{a}}_2^{\left(2\right)}\left(j_1;\mathbf{x}\right)},$$

(173)

$${\mathbf{s}}_7^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial {\mathbf{a}}_3^{\left(2\right)}\left(j_1;\mathbf{x}\right)},$$

(174)

$${\mathbf{s}}_8^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial {\mathbf{a}}_4^{\left(2\right)}\left(j_1;\mathbf{x}\right)}.$$

(175)

The adjoint boundary conditions represented by Equation (167) are selected so as to eliminate, in conjunction with the boundary conditions represented by Equation (146), all of the unknown values of the functions $\delta {\mathbf{u}}^{\left(3\right)}\left(\mathbf{x}\right)$ in the expression of the bilinear concomitant ${\left\{P^{\left(3\right)}\left[\delta {\mathbf{u}}^{\left(3\right)}\left(\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{\left({\mathsf{\alpha }}^0\right)}$, which may vanish but if it does not, it will be reduced to a residual quantity which will be denoted as ${\widehat{P}}^{\left(3\right)}\left[{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]$ and which will comprise only known values of ${\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)$, ${\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right)$, $\mathsf{\alpha }$ and $\delta \mathsf{\alpha }$.

The equations underlying the 3^rd-LASS together with the relation expressed by Equation (164) are employed in Equation (142) to obtain the following expression for the indirect-effect term ${\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta {\mathbf{u}}^{\left(3\right)}\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}$ in terms of the 3^rd-level adjoint function ${\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)$, for $j_1=1,\dots ,TP ; j_2=1,\dots ,j_1$:

$$\begin{array}{c}{\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta {\mathbf{u}}^{\left(3\right)}\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}={\left\{{〈{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right),{\mathbf{q}}_V^{\left(3\right)}\left[{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]〉}_3\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ -{\left\{{\widehat{P}}^{\left(3\right)}\left[{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\equiv {\left\{\delta R^{\left(2\right)}{\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}.\hfill \end{array}$$

(176)

As the identity in the last line of Equation (176) indicates, the dependence of the indirect-effect term ${\left\{\delta R^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{ind}$ on the variations $\delta {\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right)$ has been eliminated, by having replaced this dependency by the dependence on the 3^rd-level adjoint function ${\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)$.

Adding the expression for the indirect-effect term ${\left\{\delta R^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{ind}$ provided in Equation (176) to the expression of the direct-effect term given in Equation (142) yields the following expression for the total third-order G-differential of the response $R\left[\mathsf{\phi }\left(\mathbf{x}\right),\mathsf{\psi }\left(\mathbf{x}\right);\mathsf{\alpha }\right]$, for $for j_1=1,\dots ,TP ; j_2=1,\dots ,j_1$, which was defined in Equation (141):

$$\begin{array}{c}{\left\{\delta R^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}={\left\{\sum _{k=1}^{2^3}{〈{\mathbf{a}}_k^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right),{\mathbf{q}}_k^{\left(1\right)}\left(\mathsf{\phi };\mathsf{\alpha };\delta \mathsf{\alpha }\right)〉}_0\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ -{\left\{{\widehat{P}}^{\left(3\right)}\left[{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}+{\left\{\frac{\partial }{\partial \mathsf{\alpha }}\prod _{i=1}^{TI}\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(2\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\delta \mathsf{\alpha }\hfill \\ \triangleq \sum _{j_3=1}^{TP}{\left\{R^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\delta {\alpha }_{j_3},\hfill \end{array}$$

(177)

where

$$R^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2;j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq {\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}}$$

(178)

denotes the 3^rd-order partial sensitivity of the response with respect to the model parameters, evaluated at the nominal parameter values ${\mathsf{\alpha }}^0$.

5.4. The 4^th-CASAM-L: Summary

The fourth-order sensitivities of the response $R\left[\mathsf{\phi }\left(\mathbf{x}\right),\mathsf{\psi }\left(\mathbf{x}\right);\mathsf{\alpha }\right]$ with respect to the model parameters are obtained by determining the first-order G-differential of the 3^rd-order sensitivities $R^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\equiv {\partial }^{3}R\left(\mathsf{\phi };\mathsf{\psi };\mathsf{\alpha }\right)/\partial {\alpha }_{j_3}\partial {\alpha }_{j_2}\partial {\alpha }_{j_1}$, which are defined in Equation (178). Recalling the developments of the 3^rd-CSASM-L presented in Section 5.3, the functions ${\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right)$ and ${\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)$ are the solutions of the following system of equations, written in block-matrix form:

$${\left\{{\mathbf{F}}^{\left(4\right)}\left(\mathsf{\alpha };\mathbf{x}\right){\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)\right\}}_{{\mathsf{\alpha }}^0}= {\left\{{\mathbf{q}}_F^{\left(4\right)}\left(j_2,j_1;{\mathbf{u}}^{\left(3\right)};\mathbf{x};\mathsf{\alpha }\right)\right\}}_{{\mathsf{\alpha }}^0}, \mathbf{x}\in {\mathsf{\Omega }}_x \left({\mathsf{\alpha }}^0\right)$$

(179)

$${\left\{ {\mathbf{b}}_F^{\left(4\right)}\left[{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right); \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}=\mathbf{0}, \mathbf{x}\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right),$$

(180)

where:

$$\begin{array}{c}{\mathbf{F}}^{\left(4\right)}\left(\mathsf{\alpha };\mathbf{x}\right)\triangleq \left(\begin{array}{cc}{\left[{\mathbf{F}}^{\left(3\right)}\left(\mathsf{\alpha };\mathbf{x}\right)\right]}_{2^3\times 2^3}& \mathbf{0}\\ \mathbf{0}& {\left[{\mathbf{A}}^{\left(3\right)}\left(j_1\right)\right]}_{2^3\times 2^3}\end{array}\right);{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)\triangleq \left(\begin{array}{c}{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right)\\ {\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\end{array}\right);\hfill \\ {\mathbf{q}}_F^{\left(4\right)}\left(j_2,j_1;{\mathbf{u}}^{\left(3\right)};\mathbf{x};\mathsf{\alpha }\right)\triangleq \left(\begin{array}{c}{\mathbf{q}}_F^{\left(3\right)}\\ {\mathbf{s}}_A^{\left(3\right)}\end{array}\right);{\mathbf{b}}_F^{\left(4\right)}\left[{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\triangleq \left(\begin{array}{c}{\mathbf{b}}_F^{\left(3\right)}\\ {\mathbf{b}}_A^{\left(3\right)}\end{array}\right).\hfill \end{array}$$

(181)

The system of equations represented by Equations (179) and (180) will be called the 4^th-Level Forward System.

There are $TP\left(TP+1\right)\left(TP+2\right)\left(TP+3\right)/4!$ distinct 4^th-order sensitivities of the response with respect to the model and response parameters. In view of the definition introduced in Equation (181), the 3^rd-order sensitivity $R^{\left(3\right)}\left[j_3,j_2;j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]$ becomes a function of ${\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)$, and its total G-differential is obtained, by definition, as follows:

$$\begin{array}{c}{\left\{\delta R^{\left(3\right)}\left[j_3,j_2;j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta {\mathbf{u}}^{\left(4\right)};\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ \triangleq {\left\{\frac{d}{d\epsilon }{\left[\prod _{i=1}^{TI}\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(3\right)}\left(j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2;j_1;\mathbf{x}\right)+\epsilon \delta {\mathbf{u}}^{\left(4\right)};\mathsf{\alpha }+\epsilon \delta \mathsf{\alpha }\right)\right]}_{{\mathsf{\alpha }}^0}\right\}}_{\epsilon =0}\hfill \\ ={\left\{\delta R^{\left(3\right)}{\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)};\delta \mathsf{\alpha }\right]}_{{\mathsf{\alpha }}^0}\right\}}_{dir}+{\left\{\delta R^{\left(3\right)}{\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)};\delta {\mathbf{u}}^{\left(4\right)}\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind},\hfill \end{array}$$

(182)

where the direct-effect term ${\left\{\delta R^{\left(3\right)}{\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)};\delta \mathsf{\alpha }\right]}_{{\mathsf{\alpha }}^0}\right\}}_{dir}$ depends directly on the parameter variations and is defined as follows:

$${\left\{\delta R^{\left(3\right)}{\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)};\delta \mathsf{\alpha }\right]}_{{\mathsf{\alpha }}^0}\right\}}_{dir} \triangleq {\left\{\frac{\partial }{\partial \mathsf{\alpha }}{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}}\right\}}_{{\mathsf{\alpha }}^0}\delta \mathsf{\alpha },$$

(183)

while the “indirect-effect term” ${\left\{\delta R^{\left(3\right)}{\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)};\delta {\mathbf{u}}^{\left(4\right)}\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}$ is defined as follows:

$${\left\{\delta R^{\left(3\right)}{\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)};\delta {\mathbf{u}}^{\left(4\right)}\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}\triangleq {\left\{{\displaystyle \prod _{i=1}^{TI}{\displaystyle \underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i\frac{\partial S^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]}{\partial {\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)}}}\right\}}_{{\mathsf{\alpha }}^0}\delta {\mathbf{u}}^{\left(4\right)}$$

(184)

where

$$\begin{array}{l}\frac{\partial \left[ \right]}{\partial {\mathbf{u}}^{\left(4\right)}\left(j_2;j1;\mathbf{x}\right)}\delta {\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)\equiv \frac{\partial \left[ \right]}{\partial \mathsf{\phi }}\delta \mathsf{\phi }\left(\mathbf{x}\right)+\frac{\partial \left[ \right]}{\partial \mathsf{\psi }}\delta \mathsf{\psi }\left(\mathbf{x}\right)+{\displaystyle \sum _{k=1}^2\frac{\partial \left[ \right]}{\partial {\mathbf{a}}_k^{\left(1\right)}}}\delta {\mathbf{a}}_k^{\left(1\right)}\left(\mathbf{x}\right)\\ +{\displaystyle \sum _{k=1}^{2^2}\frac{\partial \left[ \right]}{\partial {\mathbf{a}}_k^{\left(2\right)}\left(j_1\right)}\delta {\mathbf{a}}_k^{\left(2\right)}\left(j_1;\mathbf{x}\right)+{\displaystyle \sum _{k=1}^{2^3}\frac{\partial \left[ \right]}{\partial {\mathbf{a}}_k^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)}\delta {\mathbf{a}}_k^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)} .}\end{array}$$

(185)

The vectors of variations $\delta {\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right)$ and $\delta {\mathbf{a}}^{\left(3\right)}\left(j_2;j_1;\mathbf{x}\right)$ are related to each other and must be determined by solving simultaneously the coupled system of equations obtained by concatenating the 3^rd-LVSS to the system of equations obtained by G-differentiating the 3^rd-LASS while being subject to the boundary conditions obtained by concatenating the boundary conditions which belong to the 3^rd-LVSS with the boundary conditions obtained by G-differentiating the boundary conditions belonging to the 3^rd-LASS. Concatenating all of these equations and boundary conditions yields the following 16 × 16 block-matrix equation which constitutes the 4^th-Level Variational Sensitivity System (4^th-LVSS):

$${\left[{\mathbf{V}}^{\left(4\right)}\left(j_2,j_1\right)\right]}_{2^4\times 2^4}\delta {\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)={\mathbf{q}}_V^{\left(4\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right],$$

(186)

$$\begin{array}{l}{\mathbf{b}}_V^{\left(4\right)}\left[\mathbf{u};{\mathbf{a}}^{\left(1\right)};{\mathbf{a}}^{\left(2\right)}\left(j_1\right);\mathsf{\alpha };\delta \mathsf{\phi }\left(\mathbf{x}\right);\delta \mathsf{\psi }\left(\mathbf{x}\right);\delta {\mathbf{a}}^{\left(2\right)}\left(j_1;\mathbf{x}\right);\delta \mathsf{\alpha }\right]\\ \triangleq \left(\begin{array}{c}{\mathbf{b}}_V^{\left(3\right)}\left[j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta {\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\delta \mathsf{\alpha }\right]\\ \delta {\mathbf{b}}_A^{\left(3\right)}\left[{\mathbf{u}}^{\left(2\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta {\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right) ;\delta \mathsf{\alpha }\right]\end{array}\right)=\left(\begin{array}{c}\mathbf{0}\\ \mathbf{0}\end{array}\right), \mathbf{x}\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right),\end{array}$$

(187)

where:

$${\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)\triangleq \left(\begin{array}{c}{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right)\\ {\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\end{array}\right); \delta {\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)\triangleq \left(\begin{array}{c}\delta {\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right)\\ \delta {\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\end{array}\right);$$

(188)

$$\begin{array}{c}{\left[{\mathbf{V}}^{\left(4\right)}\left(j_2,j_1\right)\right]}_{2^4\times 2^4}={\left(\begin{array}{cc}{\mathbf{V}}^{\left(3\right)}\left(j_1;{\mathbf{u}}^{\left(2\right)}\right)& {\left[\mathbf{0}\right]}_{2^3\times 2^2}\\ {\mathbf{V}}_{21}^{\left(4\right)}\left(j_2,j_1\right)& {\mathbf{V}}_{22}^{\left(4\right)}\left(j_1\right)\end{array}\right)}_{2^4\times 2^4};{\mathbf{V}}_{22}^{\left(4\right)}\left(j_1\right)\triangleq {\mathbf{A}}^{\left(3\right)}\left(j_1;{\mathbf{u}}^{\left(2\right)}\right);\hfill \\ {\mathbf{V}}_{21}^{\left(4\right)}\triangleq \frac{\partial \left\{{\mathbf{A}}^{\left(3\right)}\left[j_1;{\mathbf{u}}^{\left(2\right)}\left(j_1;\mathbf{x}\right)\right]{\mathbf{a}}^{\left(3\right)}{\left(j_2,j_1;\mathbf{x}\right)}^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}{\partial {\mathbf{u}}^{\left(3\right)}};\hfill \end{array}$$

(189)

$$\begin{array}{l}{\mathbf{q}}_V^{\left(4\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\triangleq \left(\begin{array}{c}{\mathbf{q}}^{\left(3\right)}\left[j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\\ {\mathbf{p}}^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\end{array}\right)\\ ={\left\{{\mathbf{q}}_1^{\left(4\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right],\dots ,{\mathbf{q}}_{16}^{\left(4\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}^{†},\end{array}$$

(190)

$$\begin{array}{l}{\mathbf{p}}^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\\ \triangleq \frac{\partial \left\{{\mathbf{s}}^{\left(3\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);\mathsf{\alpha }\right]-{\mathbf{A}}^{\left(3\right)}\left({\mathbf{u}}^{\left(2\right)}\right){\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\right\}}{\partial \mathsf{\alpha }}\delta \mathsf{\alpha }.\end{array}$$

(191)

The need for solving the 4^th-LVSS can be avoided by expressing the indirect-effect term ${\left\{\delta R^{\left(3\right)}\left(j_3,j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right);\delta {\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right)\right)\right\}}_{ind}$ defined in Equation (184) an alternative way, which does not involve the function $\delta {\mathbf{u}}^{\left(4\right)}\left(j_2;j_1;\mathbf{x}\right)$. This alternative expression will be obtained by using the solution of a 4^th-Level Adjoint Sensitivity Systems (4^th-LASS), which is constructed by implementing the same sequence of logical steps as were followed for constructing the 1^st, 2^nd, and 3^rd-LASS. The 4^th-LASS is constructed in a Hilbert space, denoted as ${\mathrm{H}}_4$, comprising block-vector elements of the form ${\eta }^{\left(4\right)}\left(\mathbf{x}\right)\triangleq {\left[{\eta }_1^{\left(4\right)}\left(\mathbf{x}\right),\dots ,{\eta }_{16}^{\left(4\right)}\left(\mathbf{x}\right)\right]}^{†}\in {\mathrm{H}}_4$, with ${\eta }_i^{\left(4\right)}\left(\mathbf{x}\right)\triangleq {\left[{\eta }_{i,1}^{\left(4\right)}\left(\mathbf{x}\right),\dots ,{\eta }_{i,TD}^{\left(4\right)}\left(\mathbf{x}\right)\right]}^{†}$, for $i=1,‥,16$. The inner product between two elements, ${\eta }^{\left(4\right)}\left(\mathbf{x}\right)\in {\mathrm{H}}_4$ and ${\xi }^{\left(4\right)}\left(\mathbf{x}\right)\in {\mathrm{H}}_4$, of the Hilbert space ${\mathrm{H}}_4$, will be denoted as ${〈{\eta }^{\left(4\right)}\left(\mathbf{x}\right),{\xi }^{\left(4\right)}\left(\mathbf{x}\right)〉}_4$ and is defined as follows:

$${〈{\eta }^{\left(4\right)}\left(\mathbf{x}\right),{\xi }^{\left(4\right)}\left(\mathbf{x}\right)〉}_4\triangleq {\displaystyle \sum _{i=1}^{16}{〈{\eta }_i^{\left(4\right)}\left(\mathbf{x}\right),{\xi }_i^{\left(4\right)}\left(\mathbf{x}\right)〉}_0}.$$

(192)

Using the definition provided in Equation (192), form the inner product in ${\mathrm{H}}_4$ of Equation (186) with a yet undefined vector-valued function denoted as ${\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right)\triangleq {\left[{\mathbf{a}}_1^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right),\dots ,{\mathbf{a}}_{16}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right)\right]}^{†}\in {\mathrm{H}}_4$,$j_1=1,\dots ,TP ; j_1=1,\dots ,j_2$, to obtain the following relation:

$$\begin{array}{c}{\left\{{〈{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right),{\mathbf{V}}^{\left(4\right)}\left({\mathbf{u}}^{\left(3\right)}\right)\delta {\mathbf{u}}^{\left(4\right)}〉}_4\right\}}_{{\mathsf{\alpha }}^0}={\left\{{〈{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right),{\mathbf{q}}_V^{\left(4\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right]〉}_4\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ ={\left\{{〈\delta {\mathbf{u}}^{\left(4\right)},{\mathbf{A}}^{\left(4\right)}\left(j_2;j_1\right){\mathbf{a}}^{\left(4\right)}\left(j_3,j_2;j_1;\mathbf{x}\right)〉}_4\right\}}_{{\mathsf{\alpha }}^0}+{\left\{P^{\left(4\right)}\left[\delta {\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0},\hfill \end{array}$$

(193)

where ${\left\{P^{\left(4\right)}\left[\delta {\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}$ denotes the bilinear concomitant defined on the phase-space boundary $\mathbf{x}\in \partial {\mathsf{\Omega }}_x\left({\mathsf{\alpha }}^0\right)$ and where:

$${\left[A^{\left(4\right)}\left(j_2,j_1;u^{\left(3\right)}\right)\right]}_{2^4\times 2^4}\triangleq {\left[V^{\left(4\right)}\left(u^{\left(3\right)}\right)\right]}_{2^4\times 2^4}^{*}={\left(\begin{array}{cc}{V}^{\left(3\right)}\left(u^{\left(1\right)}\right)& {\left[0\right]}_{2^3\times 2^3}\\ V_{21}^{\left(4\right)}\left(j1;u^{\left(2\right)}\right)& V_{22}^{\left(4\right)}\left(u^{\left(1\right)}\right)\end{array}\right)}^{*}=\left(\begin{array}{cc}{A}^{\left(3\right)}& {\left[V_{21}^{\left(4\right)}\right]}^{*}\\ {\left[0\right]}_{2^3\times 2^3}& {\left[V_{22}^{\left(4\right)}\right]}^{*}\end{array}\right)$$

(194)

The first term on right-side of the second equality in Equation (193) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(3\right)}\left(j_3,j_2,j_1;u^{\left(3\right)};a^{\left(3\right)};\delta u^{\left(3\right)};\delta a^{\left(3\right)}\right)\right\}}_{ind}$, which is achieved by requiring that the 4^th-level adjoint function $a^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\triangleq {\left[\dots ,a_k^{\left(4\right)}\left(j_3,j_2,j_1;x\right),\dots \right]}^{†};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^4;$ be the solution of the following 4^th-Level Adjoint Sensitivity System, for $j_1=1,\dots ,TP\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}{j}_2=1,\dots ,j_1;\hspace{0.17em}\hspace{0.17em}{j}_3=1,\dots ,j_2$:

$${\left[A^{\left(4\right)}\left(j_2,j_1;u^{\left(3\right)}\right)\right]}_{2^4\times 2^4}{a}^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\hspace{0.17em}=s_A^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\hspace{0.17em},$$

(195)

$${\left\{b_A^{\left(4\right)}\left[a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\right\}}_{{\alpha }^0}=0,\hspace{0.17em}\hspace{0.17em}x\in \partial {\Omega }_x\left({\alpha }^0\right).$$

(196)

where the vector $s_A^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\triangleq {\left[s_1^{\left(4\right)}\left(j_3,j_2,j_1;x\right),\dots ,s_{16}^{\left(4\right)}\left(j_3,j_2,j_1;x\right)\right]}^{†}$ comprises, for each $j_1=1,\dots ,TP\hspace{0.17em};\hspace{0.17em}{j}_2=1,\dots ,j_1;\hspace{0.17em}\hspace{0.17em}{j}_3=1,\dots ,j_2$, sixteen components defined as follows:

$$s_1^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\triangleq \frac{\partial S^{\left(3\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]}{\partial \phi },\hspace{0.17em}$$

(197)

$$s_2^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\triangleq \frac{\partial S^{\left(3\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]}{\partial \psi },$$

(198)

$$\hspace{0.17em}{s}_{2+k}^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\triangleq \frac{\partial S^{\left(3\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]}{\partial a_k^{\left(1\right)}};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,2;$$

(199)

$$\hspace{0.17em}{s}_{4+k}^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\triangleq \frac{\partial S^{\left(3\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]}{\partial a_k^{\left(2\right)}\left(j_1\right)};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^2;$$

(200)

$$\hspace{0.17em}{s}_{8+k}^{\left(4\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha \right]\triangleq \frac{\partial S^{\left(3\right)}\left[j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1,x\right);\alpha \right]}{\partial a_k^{\left(3\right)}\left(j_2,j_1\right)};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,\dots ,2^3\hspace{0.17em}.$$

(201)

The 4^th-level adjoint boundary conditions represented by Equation (196) are selected so as to eliminate, in conjunction with the boundary conditions represented by Equation (187), all of the unknown values of the functions $\delta u^{\left(4\right)}\left(j_3,j_2,j_1;x\right)$ in the expression of the bilinear concomitant ${\left\{P^{\left(4\right)}\left[\delta u^{\left(4\right)}\left(j_2,j_1;x\right);a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$. This bilinear concomitant may vanish after implementing the boundary conditions represented by Equations (196) and (187); if it does not vanish, it will be reduced to a residual quantity which will comprise only known values of $a^{\left(4\right)}\left(j_3,j_2,j_1;x\right)$, $u^{\left(4\right)}\left(j_2,j_1;x\right)$, $\alpha$ and $\delta \alpha$, which will be denoted as ${\left\{{\widehat{P}}^{\left(4\right)}\left[a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);u^{\left(4\right)}\left(j_2,j_1;x\right);\alpha ;\delta \alpha \right]\right\}}_{{\alpha }^0}$.

Using the equations underlying the 4^th-LASS together with the relation provided in Equation (193) in Equation (184) yields the following expression for the indirect-effect term ${\left\{\delta R^{\left(3\right)}\left[j_3,j_2,j_1;u^{\left(3\right)}\left(j_1;x\right);a^{\left(3\right)}\left(j_2,j_1;x\right);\delta u^{\left(3\right)}\left(j_1;x\right);\delta a^{\left(3\right)}\left(j_2,j_1;x\right)\right]\right\}}_{ind}$ in terms of the 4^th-level adjoint functions $a^{\left(4\right)}\left(j_3,j_2,j_1;x\right)$, for $j_1=1,\dots ,TP\hspace{0.17em};j_2=1,\dots ,j_1;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{j}_3=1,\dots ,j_2$:

$$\begin{array}{c}\left\{\delta R^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\right.\right.;\delta {\mathbf{u}}^{\left(3\right)}\left(j_1;\mathbf{x}\right);{\left.{\left.\delta {\mathbf{a}}^{\left(3\right)}\left(j_2,j_1;\mathbf{x}\right)\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}\hfill \\ =-{\left\{{\widehat{P}}^{\left(4\right)}\left[{\mathbf{a}}^{\left(4\right)};{\mathbf{u}}^{\left(4\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}+{\left\{{〈{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right),{\mathbf{q}}_V^{\left(4\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]〉}_4\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ \equiv {\left\{\delta R^{\left(3\right)}{\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]}_{{\mathsf{\alpha }}^0}\right\}}_{ind}.\hfill \end{array}$$

(202)

Adding the result obtained in Equation (202) for the indirect effect term to the result in Equation (183) for the direct effect term yields the following expression for the total 3^rd-order G-variation defined in Equation (182):

$$\begin{array}{c}{\left\{\delta R^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha };\right]\right\}}_{\left({\mathsf{\alpha }}^0\right)}\hfill \\ ={\left\{\frac{\partial }{\partial \mathsf{\alpha }}\prod _{i=1}^{TI}\underset{{\lambda }_i\left(\mathsf{\alpha }\right)}{\overset{{\omega }_i\left(\mathsf{\alpha }\right)}{\int }}d{x}_i{S}^{\left(3\right)}\left[j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}_{{\alpha }^0}\delta \alpha ,\hfill \\ -{\left\{{\widehat{P}}^{\left(4\right)}\left[{\mathbf{a}}^{\left(4\right)};{\mathbf{u}}^{\left(4\right)};\mathsf{\alpha };\delta \mathsf{\alpha }\right]\right\}}_{{\mathsf{\alpha }}^0}+{\left\{{〈{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right),{\mathbf{q}}_V^{\left(4\right)}\left[j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);\mathsf{\alpha };\delta \mathsf{\alpha }\right]〉}_4\right\}}_{{\mathsf{\alpha }}^0}\hfill \\ \triangleq \sum _{j_4=1}^{TP}{\left\{R^{\left(4\right)}\left[j_4,j_3,j_2,j_1;{\mathbf{u}}^{\left(4\right)}\left(j_2,j_1;\mathbf{x}\right);{\mathbf{a}}^{\left(4\right)}\left(j_3,j_2,j_1;\mathbf{x}\right);\mathsf{\alpha }\right]\right\}}_{\left({\mathsf{\alpha }}^0\right)}\delta {\alpha }_{j_4},\hfill \end{array}$$

(203)

where $R^{\left(4\right)}\left[j_4,j_3,j_2,j_1;u^{\left(4\right)}\left(j_2,j_1;x\right);a^{\left(4\right)}\left(j_3,j_2,j_1;x\right);\alpha \right]$ denotes the 4^th-order partial sensitivity of the response with respect to the model parameters.

6. Discussion and Conclusions

This work has presented the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Linear Systems” (abbreviation: n^th-CASAM-L), where “n” is a finite number that indicates any desired, arbitrarily-high, order. The n^th-CASAM-L enables the efficient computation of the exact expressions of the n^th-order functional derivatives (“sensitivities”) of a general system response, which depends on both the forward and adjoint state functions, with respect to all of the parameters (including boundary and initial conditions) underlying the respective forward and adjoint systems. Since nonlinear operators do not admit adjoint operators (only linearized versions of nonlinear operators can admit a bona-fide adjoint operator), responses that simultaneously depend on forward and adjoint functions can arise only in conjunction with linear systems. Particularly important model responses that involve both the forward and adjoint functions are various forms of Lagrangian functionals, the Roussopoulos functional for computing reaction rates, the Raleigh quotient for computing eigenvalues and/or separation constants when solving partial differential equations, the Schwinger functional for “normalization-free” solutions, and many others, (e.g., [1,2]). These functionals play a fundamental role in various optimization and control procedures, derivation of numerical methods for solving (differential, integral, integro-differential) equations, etc. The sensitivity analysis of responses that simultaneously involve both forward and adjoint state functions makes it necessary to treat linear models/systems in their own right, rather than treating them as particular cases of nonlinear systems (in which the responses can depend only on the forward functions).

This work has shown that the mathematical framework underlying the n^th-CASAM-L is set in linearly increasing higher-dimensional Hilbert spaces, as opposed to exponentially increasing parameter-dimensional spaces. In particular, for a scalar-valued valued response associated with a nonlinear model comprising TP parameters, the 1^st-CASAM-L requires 1 additional large-scale adjoint computation (as opposed to TP large-scale computations, as required by other methods) for computing exactly all of the 1^st-order response sensitivities. All of the (mixed) 2^nd-order sensitivities are computed exactly by the 2^nd-CASAM-L in at most TP computations, as opposed to TP(TP + 1)/2 computations required by all other methods, and so on. For every lower-order sensitivity of interest, the n^th-CASAM-L computes the “TP next-higher-order” sensitivities in one adjoint computation performed in a linearly increasing higher-dimensional Hilbert space. Very importantly, the n^th-CASAM-L computes the higher-level adjoint functions using the same forward and adjoint solvers (i.e., computer codes) as used for solving the original forward and adjoint systems, thus requiring relatively minor additional software development for computing the various-order sensitivities.

The n^th-CASAM presented in this work is the only practically implementable methodology for obtaining the exact expressions (i.e., free of methodologically-introduced approximations) of arbitrarily-high order sensitivities (functional derivatives) of model responses to model parameters, for coupled forward/adjoint linear systems. By enabling the practical computation of any arbitrarily-order response sensitivities to model parameters, the n^th-CASAM-L makes it possible to compare the relative values of the sensitivities of various order, in order to assess which sensitivities are important and which may actually be neglected, thus enabling future investigations of the convergence of the (multivariate) Taylor series expansion of the response in terms of parameter variations, as well as investigating the actual validity of expressions that are derived from Taylor-expansion of the response (e.g., response variances/covariance, skewness, kurtosis, etc.) as a function of the model’s parameters. The larger the number of model parameters, the more efficient the C-ASAM-L becomes for computing arbitrarily high-order response sensitivities. The n^th-CASAM-L also enables the direct derivation, on paper, of the expression of a specific high-order sensitivity of interest, which can subsequently be computed directly; such a direct computation is not possible with any statistical method.

The n^th-CASAM-L presented in this work provides a fundamentally important step in the quest to overcome the “curse of dimensionality” in sensitivity analysis, uncertainty quantification and predictive modeling. Ongoing work aims at developing the “n^th-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (abbreviation: n^th-CASAM-N) for the practical, efficient, and exact computation of arbitrarily-high order sensitivities of responses to model parameters for nonlinear systems. The n^th-CASAM-L, together with the n^th-CASAM-N, are expected to revolutionize all of the fields of activities which require response sensitivities, including the fields of uncertainty quantification, model validation, optimization, data assimilation, model calibration, sensor fusion, reduced-order modeling, inverse problems, and predictive modeling.