1. Introduction
The accompanying Part I [
1] has reviewed the need for computing high-order sensitivities and has highlighted their subsequent uses for performing sensitivity analyses of model responses to model parameters, quantification of uncertainties induced in responses by uncertainties in parameters, and their essential contributions to high-order predictive modeling—which combines computational with experimental information to obtain best-estimate predicted responses and calibrated model parameters with reduced predicted uncertainties.
For large-scale models involving many parameters, even the first-order sensitivities are computationally very expensive to determine accurately by conventional methods. Furthermore, the “curse of dimensionality” [
2] prohibits the accurate computation of higher-order sensitivities by conventional methods. Cacuci [
3] has developed the “nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (abbreviated as “nth-CASAM-L”), which overcomes the curse of dimensionality in sensitivity analysis for such systems. The “nth-CASAM-L” methodology was developed specifically for linear systems because the most important model responses produced by such systems are various Lagrangian functionals which depend simultaneously on both the forward and adjoint state functions governing the respective linear system. Since nonlinear operators do not admit adjoint operators, responses that simultaneously depend on both the forward and the adjoint state functions can occur only for linear systems. Cacuci [
4] has also extended his original work [
5,
6] on nonlinear systems by developing the “nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems” (abbreviated as: nth-CASAM-N). Just like the nth-CASAM-L [
3], the nth-CASAM-N [
4] is formulated in linearly increasing higher-dimensional Hilbert spaces (as opposed to exponentially increasing parameter-dimensional spaces), thus overcoming the curse of dimensionality in sensitivity analyses of nonlinear systems. The nth-CASAM-L and the nth-CASAM-N methodologies enable the most efficient computation of exactly-determined expressions of arbitrarily high-order sensitivities of generic linear and/or nonlinear system responses with respect to model parameters, uncertain boundaries, and uncertain internal interfaces in the model’s phase-space. Additional details regarding applications of these high-order sensitivity analysis methodologies are provided in [
7,
8].
The nth-CASAM-L and the nth-CASAM-N methodologies compute the sensitivities of model responses directly with respect to the model’s primary parameters. However, many models comprise not just disparate primary model parameters but also functions of such primary model parameters. Such functions customarily appear in models in the form of correlations that describe “features” of the system under consideration, such as material properties, flow regimes, etc. Usually, the number of such “feature” functions is considerably smaller than the total number of model parameters. For example, the numerical model of the OECD/NEA reactor physics benchmark analyzed in [
9] comprises 21,976 uncertain primary model parameters. However, in the large-scale numerical computation which solves the Boltzmann neutron transport equation to compute the neutron flux distribution within this benchmark, the computational model [
9] does not use the primary parameters directly, but uses various functions of these parameters. High-order sensitivity analyses and uncertainty quantification investigations [
9] have shown that the most important of these functions are the 180 group-averaged total macroscopic cross sections, which are functions/features of the microscopic cross sections and isotopic number densities, which in turn are uncertain primary model parameters. However, a methodology for computing exact expressions of response sensitivities to functions/features of the primary model parameters is not yet available. It is the purpose of this work to introduce, for the first time, such a methodology, which enables the most efficient computation of exactly obtained expressions of sensitivities of model responses to functions/features of model parameters. This methodology will be called the “High-Order Functions/Features Adjoint Sensitivity Analysis Methodology” (nth-FASAM).
This work is structured as follows:
Section 2 presents the general mathematical modeling of a generic nonlinear system comprising functions/features of uncertain parameters and boundaries.
Section 3 introduces the mathematical framework of the “First-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (1st-FASAM).
Section 4 illustrates the application of the 1st-FASAM to determine the first-order sensitivities of a “dosimetry” detector response for a paradigm model of particle (neutrons and/or gamma rays) transport through a shielding material. This benchmark is representative of large-scale computations, such as those performed for the OECD/NEA reactor physics benchmark mentioned above. This paradigm particle transport model admits closed-form analytical solutions for the sensitivities of the model’s response with respect to the model’s “feature” functions, thereby highlighting the efficiency of the 1st-FASAM in obtaining exact expressions for these sensitivities and computing the respective expressions most efficiently. Comparisons with the 1st-CASAM [
7] are also provided, illustrating the transition from the 1st-FASAM (which provides exact sensitivities to functions/features of parameters) to the connection with the 1st-CASAM (which provides responses sensitivities directly with respect to the primary parameters).
Section 5 introduces the mathematical framework of the “Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (2nd-FASAM). It is shown that the 2nd-FASAM solves the same 2nd-Level adjoint Sensitivity System (2nd-LASS) as the 2nd-CASAM, but there are only as many (large-scale) adjoint computations as there are “features”, as opposed to as many adjoint computations as there are primary parameters, which is the case for the 2nd-CASAM. Thus, there are considerably fewer large-scale computations within the framework of the 2nd-FASAM, since the transition from the sensitivities with respect to feature functions to the sensitivities with respect to the model’s primary parameters necessitates just trivial (as opposed to large-scale) computations.
Section 6 illustrates the application of the 2nd-FASAM for computing the second-order sensitivities for the dosimetry-response of the same paradigm particle transport model as was analyzed in
Section 4, confirming the unmatched efficiency of the 2nd-FASAM for computing exact expressions of second-order sensitivities of model responses to both functions of parameters and to the primary parameters themselves. The discussion presented in
Section 7 concludes this work, noting that ongoing research aims at developing the general framework (n
th-FASAM) for computing arbitrarily high-order sensitivities of responses to functions/features of parameters, which will thus provide the most efficient methodology for computing exact response sensitivities to model parameters.
2. Modeling of a Generic Nonlinear System Comprising Functions (“Features”) of Uncertain Parameters and Boundaries
The mathematical model that underlies the numerical evaluation of a process and/or state of a physical system comprises equations that relate the system’s independent variables and parameters to the system’s state/dependent variables. These coupled equations, which are in general nonlinear, can be represented generically in operator form as follows:
The results computed using a mathematical model are customarily called “model responses” (or “system responses” or “objective functions” or “indices of performance”). As has been discussed in [
7,
8], all responses can be fundamentally analyzed in terms of the following generic integral representation:
where
is a suitably differentiable nonlinear function of
and of
.
Without loss of generality, the quantities which appear in Equations (1)–(3) can be considered to be real-valued, and have the following meanings:
1. Matrices are denoted using capital bold letters while vectors will be denoted using either capital or lower-case bold letters. The symbol “” will be used to denote “is defined as” or “is by definition equal to.” Transposition will be indicated by a dagger superscript. The equalities in this work are considered to hold in the weak (“distributional”) sense. Both sides of Equations (1)–(3), respectively, may contain “generalized functions/functionals”, particularly Dirac-distributions and derivatives thereof.
2. The -dimensional column-vector represents the “vector of primary model parameters” and has components denoted as ,…,, where denotes the “total number of parameters” involved in the model under consideration. These model parameters usually stem from processes that are external to the system under consideration and are seldom, if ever, known precisely. The known characteristics of the model parameters usually include their nominal (expected/mean) values and, possibly, higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis), which are usually determined from experimental data and/or processes external to the physical system under consideration. Occasionally, just the lower and the upper bounds may be known for some model parameters, expressed by inequality and/or equality constraints that delimit the ranges of the system’s parameters which are known. Without loss of generality, the imprecisely known model parameters can be considered to be real-valued scalar quantities. It is important to note that the components of the vector include not only parameters that appear in Equations (1) and (2), but also include parameters that may specifically occur only in the definition of the model’s response provided in Equation (3). The nominal parameter values will be denoted as ; the superscript “0” will be used throughout this monograph to denote “nominal values.”
3. The -dimensional column vector comprises the model’s independent variables, denoted as ; the sub/superscript “TI” denotes the “total number of independent variables”. The vector is considered to be defined on a phase-space domain denoted as , including the particular cases when for some independent variables . The domain boundary of is defined to comprise the set of all of the endpoints . For subsequent mathematical developments, it is convenient to consider that the endpoints are components of column-vectors and , respectively These endpoints depend on the physical system’s geometrical dimensions, which may be imprecisely known because of manufacturing tolerances, and are considered therefore to be components of the vector of primary model parameters. Furthermore, the boundary-endpoints may also depend on the parameters that define the material properties of the respective medium. For example, in models based on diffusion theory, the boundary conditions for materials facing air/vacuum are imposed on a physics-based mathematical construct called the “extrapolated boundary” of the respective spatial domain. The “extrapolated boundary” depends both on the imprecisely known physical dimensions of the system’s materials and also on the materials’ properties, such as atomic number densities and microscopic transport cross sections. Therefore, the boundary end-points can be considered, in general, to be functions of (some of) the primary model parameters.
4. The -dimensional column vector comprises the model’s dependent variables (also called “state functions”) ; the abbreviation “” denotes “total number of dependent variables”.
5. The vector
is a
-dimensional vector having components
, which are real-valued functions of (some of) the primary model parameters
. Such functions customarily appear in models in the form of correlations that describe “features” of the system under consideration, such as material properties, flow regimes, etc. Usually, the number of functions
is considerably smaller than the total number of model parameters, i.e.,
. For example, the numerical model [
9] of the OECD/NEA reactor physics benchmark comprises 21,976 uncertain primary model parameters (including microscopic cross sections and isotopic number densities) but the neutron transport equation, which is solved to determine the neutron flux distribution within the benchmark, does not use these parameters directly but instead uses “group-averaged macroscopic cross sections”, which are functions/features of the microscopic cross sections and isotopic number densities (which in turn are uncertain quantities that would be components of the vector of primary model parameters). In particular, a component
may simply be one of the primary model parameters
, i.e.,
.
6. The -dimensional column vector comprises components , which are operators (including differential, difference, integral, distributions, and/or finite or infinite matrices) acting nonlinearly (in general) on the dependent variables , the independent variables and on the functions of model parameters .
7. The -dimensional column vector , having components , denotes inhomogeneous source terms, which usually depend nonlinearly on the uncertain parameters .
8. The components of are nonlinear operators, while the components of represent inhomogeneous boundary sources, all defined on the boundary .
9. The integral representation of the response provided in Equation (3) can represent “averaged” as well as “point-valued” quantities in the phase-space of independent variables. For example, if
represents the computation or the measurement (which would be a “detector-response”) of a quantity of interest at a point
in the phase-space of independent variables, then
would contain a Dirac-delta functional of the form
. Responses that represent “differentials/derivatives of quantities” would contain derivatives of Dirac-delta functionals in the definition of
. The vector
, having components
, which appears among the arguments of the function
, represents functions of primary parameters that often appear solely in the definition of the response but do not appear in the mathematical definition of the model, i.e., in Equations (1) and (2). The quantity
denotes the total number of such functions which appear exclusively in the definition of the model’s response. Evidently, the response will depend directly and/or indirectly (through the “feature”-functions) on all of the primary model parameters. This fact has been indicated in Equation (3) by using the vector-valued function
as an argument in the definition of the response
to represent the concatenation of all of the “features” of the model and response under consideration. The vector
is thus defined as follows:
Solving Equations (1) and (2) at the nominal parameter values,
, provides the “nominal solution”
, i.e., the vectors
and
satisfy the following equations:
Using the nominal parameter values
together with the “nominal solution”
in Equation (3) yields the nominal value of the response, namely:
In view of Equation (7), model responses
, where
denotes the “total number of responses”, can be considered to depend directly on the functions
, and would therefore admit a Taylor-series expansion around the nominal value
, having the following form:
where
. The “sensitivities of the model response with respect to the (feature) functions” are naturally defined as being the
functional derivatives of
with respect to the components (“features”)
of
. Since
, the computations of the functional derivatives of
with respect to the functions
, which appear in Equation (8), will be considerably less expensive computationally than the computation of the functional derivatives involved in the Taylor-series of the response with respect to the model parameters. The functional derivatives of the response with respect to the parameters can be obtained from the functional derivatives of the response with respect to the “feature” functions
by simply using the chain rule, i.e.:
and so on. The evaluation/computation of the functional derivatives
,
, etc., does not require computations involving the model, and is therefore trivial by comparison to the evaluation of the functional derivatives (“sensitivities”) of the response with respect to either the functions (“features”)
or the model parameters
.
The range of validity of the Taylor-series shown in Equation (8) is defined by its radius of convergence. The accuracy—as opposed to the “validity”—of the Taylor-series in predicting the value of the response at an arbitrary point in the phase-space of model parameters depends on the order of sensitivities retained in the Taylor-expansion: the higher the respective order, the more accurate the respective response value predicted by the Taylor-series. In the particular cases when the response happens to be a polynomial function of the “feature” functions , the Taylor series is actually exact.
In turn, the functions
can also be formally expanded in a multivariate Taylor-series around the nominal (mean) parameter values
, namely:
The domain of validity of the Taylor-series in Equation (10) is defined by its own radius of convergence.
In practice, the model parameters are not known exactly. Even though these parameters are not bona fide random quantities, they are formally considered to be variates that obey a multivariate probability distribution function, denoted as
, which is usually unknown. The various moments of
are defined in a standard manner by using the following formal definition for the “expected (or mean) value” of a function,
, of the parameters
, which is defined over the domain of definition of
:
In particular, the vector of expected values (denoted as
) of the model parameters
is defined as follows:
The covariances,
, between two parameters,
and
, are defined as follows:
In Equation (13), the quantities and denote the standard deviations of and , respectively, while denotes the correlation between the respective parameters.
The third-order correlation,
, among three parameters is defined as follows:
The fourth-order correlation
among four parameters is defined as follows:
In terms of the moments of the parameter distribution, the moments of the distribution of the function in the phase-space of model parameters are obtained by using the Taylor-series provided in Equation (10) in conjunction with the definitions provided in Equations (12)–(15) to obtain the following expressions:
- (i)
The mean (or expected value) of
:
- (ii)
the covariance
of two functions
and
:
Higher-order moments of the distribution of in terms of the moments of the distribution of the model parameters are obtained similarly.
In terms of the moments of the distribution of
, the distribution of the responses
is obtained by using the Taylor-series shown in Equation (8). Thus, in terms of the moments of the distribution of
, the expectation value
of
will have the following expression:
The covariance
of two responses
and
will have the following expression:
3. Introducing the 1st-FASAM: First-Order Function/Feature Adjoint Sensitivity Analysis Methodology
The “First-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (1st-FASAM), which will be developed in this section, will enable the most efficient computation of model response sensitivities to the components of the “features” function of model parameters. The nominal (or mean) parameter values, , are considered to be known, but these values will differ from the true values , which are unknown, by variations , where . The parameter variations will induce variations in the vector-valued function and will also induce variations around the nominal solution in the forward state functions, since the forward state functions are related to through Equations (1) and (2). In turn, the variations and will induce variations in the system’s response .
Mathematically, the quantity
is the first-order Gateaux (G)-variation of the response
, induced by arbitrary variations
in a neighborhood around the nominal functions and parameter values
, and is defined as follows:
where
is a scalar quantity. The G-variation
is an operator defined on the same domain as
and has the same range as
. The existence of the G-variation
does not guarantee its numerical computability. Numerical methods most often require that
be linear in
in a neighborhood
around
, thereby admitting a total first-order
G-derivative, which will be denoted as
. The necessary and sufficient conditions for a nonlinear operator
to admit a total G-differential in a neighborhood
around
are as follows:
- (i)
satisfies a weak Lipschitz condition at
, namely:
- (ii)
satisfies the following condition for a scalar
and vectors
:
In practice, the relations provided in Equations (21) and (22) are seldom used directly, since the computation of the expression on the right-hand side of Equation (20) reveals immediately if the respective expression is linear (or not) in the vectors and/or .
Numerical methods (e.g., Newton’s method and variants thereof) for solving Equations (1)and (2) also require the existence of the first-order G-derivatives of the original model equations. Therefore, the conditions provided in Equations (21) and (22) are henceforth considered to be satisfied by the model responses and also by the operators underlying the physical system modeled by Equations (1)–(3), which implies that all of the operators/functions considered in this work admit G-derivatives.
When the 1st-order G-variation
satisfies the conditions provided in (21) and (22), it can be written as follows:
In Equation (23), the “direct-effect” term
comprises only dependencies on
and is defined as follows:
where:
The notation on the left-hand side of Equation (25) represents an inner product between two vectors (so it comprises an implied multiplication of a vector with a transposed vector), but the “dagger” () which indicates “transposition” has been omitted, in order to keep the notation as simple as possible. “Daggers” indicating transposition will also be omitted in other inner products, whenever possible, while avoiding ambiguities.
The direct-effect term can be computed after having solved Equations (1) and (2) to obtain the nominal values,
, of the dependent variables. On the other hand, the quantity
defined in Equation (23) comprises only variations in the state functions—and is therefore called the “indirect-effect term”—and is defined as follows:
where
The “indirect-effect” term induces variations in the response through the variations in the state functions, which are, in turn, caused by the parameter variations through the equations underlying the model. The indirect-effect term can be quantified only after having determined the variations in terms of the variations , , and .
The first-order relationship between the vectors
and the variations
,
, and
is determined by solving the equations obtained by applying the definition of the G-differential to Equations (1) and (2), which yields the following equations:
Carrying out the differentiations with respect to
in Equations (32) and (33), and setting
in the resulting expressions yields the following equations:
In Equations (34) and (35), the superscript “(1)” indicates “1st-Level” and the various quantities which appear in these equations are defined as follows:
The system of equations comprising Equations (34) and (35) is called the “1st-Level Variational Sensitivity System” (1st-LVSS). The solution, , of the 1st-LVSS will be a function of the variations , , and . Hence, when the function is introduced into the expression of the indirect-effect term defined in Equation (30), it will introduce dependencies of the response sensitivities on , , and , which will be in addition to the dependencies displayed by the direct-effect term defined in Equation (24). As Equations (34) and (35) indicate, every parameter variation , , will induce, directly or indirectly, a variations in the model’s state variables. In principle, therefore, for every parameter variation , , there would be a corresponding solution , , of the 1st-LVSS. Thus, if the effect of every parameter variation were of interest, then the 1st-LVSS would need to be solved times, with distinct right-hand sides and boundary conditions for each parameter variation , which would require at least large-scale computations. If only variations induced by the functions , , and/or were of interest, then fewer than large-scale computations would be required.
However, solving the 1st-LVSS can be avoided altogether by using the ideas underlying the “adjoint sensitivity analysis methodology” as originally conceived by Cacuci [
5,
6] and subsequently generalized by Cacuci [
7,
8] to enable the computation of arbitrarily high-order response sensitivities to model parameters. Thus, the need for computing the vectors
,
is eliminated by expressing the indirect-effect term defined in Equation (30) in terms of the solutions of the “1st-Level Adjoint Sensitivity System” (1st-LASS), which is constructed by introducing a (real) Hilbert space denoted as
, endowed with an inner product of two vectors
and
which is denoted as
and defined as follows:
In Equation (39), the “dagger” (), which indicates “transposition,” has been omitted to simplify the notation for the scalar product .
The 1st-LASS is now constructed by considering a vector
, which is an element in
but is otherwise arbitrary at this stage, and by using Equation (39) to form the inner product of
with the relation provided in Equation (34) to obtain:
The left-hand side of Equation (40) is transformed by using the definition of the adjoint operator in
, as follows:
where
denotes the associated bilinear concomitant evaluated on the domain’s boundary
and where
denotes the operator formally adjoint to
. The symbol
indicates “formal adjoint” operator.
The first term on the right-hand side of Equation (41) is now required to represent the indirect-effect term defined in Equation (30) by imposing the following relationship:
The domain of
is determined by selecting the appropriate adjoint boundary and/or initial conditions, which will be denoted in operator form as:
The above boundary conditions for the adjoint operator
are obtained by imposing the following requirements: (i) they must be independent of unknown values of
and
; and (ii) the substitution of the boundary and initial conditions represented by Equations (35) and (43) into the expression of
must cause all terms containing unknown values of
to vanish. Using the adjoint and forward variational boundary conditions represented by Equations (43) and (35) into (41) reduces the bilinear concomitant
to a residual term denoted as
, which will contain boundary terms involving only known values of
,
;
,
, and
The residual term
is linear in
,
, and can therefore be expressed in the following form:
The results obtained in (41) and (42) are now replaced in (30) to obtain the following expression of the indirect-effect term as a function of
:
Replacing in Equation (24) the result obtained in Equation (45), together with the expression for the direct-effect term provided in Equation (26), yields the following expression for the first-order G-differential of the response
:
where
denotes the first-order sensitivity of the response
with respect to the components
of the “feature” function
. Each sensitivity
is obtained by identifying the expression that multiplies the corresponding component of the variations
,
,
, and
, respectively, on the right-hand side of Equation (46). Each of the 1st-order sensitivities
of the response
with respect to the components of the functions
,
,
, and
can be computed inexpensively after having obtained the 1st-level adjoint function
, using just quadrature formulas to evaluate the inner products involving
in Equation (46). The function
is obtained by solving numerically (42) and (43), which is the only large-scale computation needed for obtaining all of the first-order sensitivities.
Equations (42) and (43) are called the
1st-Level Adjoint Sensitivity System (1st-LASS). The solution,
, of the 1st-LASS is called the 1st-level adjoint function. It is very important to note that the 1st-LASS is independent of all parameter variations
,
,
, and
(or, equivalently, of all parameter variations
,
) and therefore needs to be solved only once, regardless of the number of model parameters under consideration. Furthermore, since the 1st-LASS is linear in
, solving it requires less computational effort than solving the original model, which is nonlinear in
. The first-order sensitivity
can be represented formally in the following integral form:
The functions will be subsequently used for determining the exact expressions of the second-order sensitivities of the response with respect to the components of the function of model parameters.
3.1. Comparison between 1st-FASAM and 1st-CASAM
The general methodology for computing most efficiently the exactly-derived expressions of the first-order sensitivities of a model response
directly with respect to the model’s parameters is the “First-Order Comprehensive Adjoint Sensitivity Analysis Methodology” (1st-CASAM) developed by Cacuci [
5,
6,
7,
8], which was briefly reviewed in the accompanying Part I [
1]. The conceptual difference between the 1st-CASAM and the 1st-FASAM presented in the first work stem from the fact that within the 1st-CASAM the possible functions (“features”) of model parameters are not explicitly considered, so that all differentials are computed directly with respect to the basic model parameters. Consequently, the main distinctions between the 1st-CASAM and the 1st-FASAM are as follows:
- (i)
Instead of the expressions provided in Equations (24)–(29), the “direct-effect” term in the 1st-CASAM, denoted as
, comprises the direct dependencies on
and is defined as follows:
where
denotes the partial G-derivatives of
with respect to
, evaluated at the nominal parameter values, and where the following definitions were used:
- (ii)
Instead of Equations (34) and (35), the 1st-LVSS within the 1st-CASAM framework has the following expression:
where:
To determine the solutions of the 1st-LVSS that would correspond to every parameter variation , , the 1st-LVSS would need to be solved times, with distinct right-hand sides for each , thus requiring large-scale computations.
- (iii)
The first-order G-differential of the response induced by variations in the model parameters and, consequently, the first-order sensitivities
of the response with respect to the model parameters have the following expressions:
where, for each
, the quantity
denotes the 1st-order sensitivities of the response
with respect to the model parameters
and has the following expression, for
:
Comparing Equation (56) to Equation (46) indicates that the same first-level adjoint function is used in all of these expressions. Therefore, the 1st-LASS needs to be solved only once, to obtain , regardless of whether the response sensitivities are computed directly with respect to the model parameters or whether the sensitivities are first computed with respect to the components (“features”) of the function and subsequently computed with respect to the components of the vector of model parameters. This is as expected, since the 1st-LASS is independent of parameter variations ,, and therefore needs to be solved only once, regardless of the number of “features” or model parameters under consideration.
It is more advantageous to use the 1st-FASAM to compute the sensitivities of the response to the “feature” functions provided in Equation (46), since these expressions involve fewer integrals (“quadrature”) over the 1st-level adjoint function
than using Equation (56), since
. The transition from the sensitivities with respect to “features” provided in Equation (46) to the sensitivities with respect to the model parameters provided in Equation (56) can be carried out very inexpensively by using the “chain-rule” of differentiation of compound functions, namely
,
,
,
, which implies that:
The features of the 1st-FASAM will be illustrated in
Section 5, below, by means of a paradigm neutron diffusion model which admits closed-form exact expressions for all quantities involved (i.e., forward dependent variable, model response, and all sensitivities). The considerable reduction in the number of large-scale computations when using the “function/feature adjoint sensitivity analysis methodology” will become apparent for the computation of second-order (and, subsequently, higher-order) sensitivities, as will be shown in
Section 5 and
Section 6, below.
4. Illustrative Application of the 1st-FASAM to a Paradigm Particle Transport Model
The application of the 1st-FASAM will be illustrated in this section by considering the sensitivity analysis of an illustrative computational model of a dosimetric response (e.g., energy deposition) stemming from a shielded volumetric source of directly and/or indirectly ionizing radiation. Such models typically describe the radiation and/or particle transport within shielded containers/drums holding vitrified radioactive waste material for deep underground disposal. The distribution of gamma-ray photons emitted by waste products within a shielded vitrified container is computed using sophisticated time-dependent three-dimensional energy-dependent radiation decay and transport models. To illustrate the application of the 1st-FASAM, however, the radiation transport model will be greatly simplified by considering that all processes are steady-state, the radiation is monoenergetic, the sources of radiation are additive, and the geometry is one-dimensional (slab) rather than three-dimensional. These drastic simplifications will enable the analytical solving of the transport equation, thereby minimizing cumbersome algebraic manipulations while highlighting the application of the fundamental concepts underlying the 1st-FASAM.
Consider that a total of “
MQ” radioactive sources, each emitting
particles per unit time (e.g., “second”) per unit area (e.g., “cm
2”) are homogeneously distributed (vitrified) within a slab of homogeneous material, which will be designated as “Material A”. There are many radioactive sources (i.e.,
MQ is a large number); even after several years of “cooling” there remain long-lived radionuclides which require adequate shielding. Included among the most relevant for shielding of waste disposal containers are 11 fission products, 12 actinides, and 5 activation products [
10]. The vitrified containment material, designated as Material A, is considered to be homogeneous, comprising “
MA” nuclides/elements. Each of these nuclides is characterized by a total microscopic cross section denoted as
,
, and an isotopic number density denoted as
,
, where
denotes the “total number of elements/nuclides within the glassy “Material A”. Geometrically, consider that Material A is a slab of a thickness
2a (cm), which is imprecisely known due to manufacturing tolerances. The slab of Material A, which contains radioactive sources, is considered to be shielded on each of its sides by another slab, made of “Material B”, of thickness
b (cm), which is also imprecisely known due to manufacturing tolerances. “Material B” is considered to comprise a total number of
nuclides/elements, each of which is characterized by a total microscopic cross section denoted as
,
, and an isotopic number density denoted as
,
. For example, “Material B” could be magnetite concrete (approximate density: 3.53 g/cm
3), composed of
nuclides/elements [
11], with approximate partial densities in (g/cm
3) provided in parentheses, as follows: 1. Hydrogen (0.011); 2. Oxygen (1.168); 3. Silicon (0.091); 4. Calcium (0.251); 5. Magnesium (0.033); 6. Aluminum (0.083); 7. Sulfur (0.005); 8. Iron (1.676); 9. Titanium (0.192); 10. Chromium (0.006); 11. Manganese (0.007); and 12. Vanadium (0.011).
The distribution of monoenergetic particles within the aforementioned arrangement of slabs is symmetrical with respect to the midplane of the inner slab, so the center of the coordinate system can be chosen at this midplane. Within the inner slab, the uncollided angular flux of monoenergetic neutral particles (e.g., photons and/or neutrons), denoted as
, in the positive direction perpendicular to the slab, labeled as “direction
”, is modeled by the following form of the particle transport equation:
In Equation (58), the source term is defined as follows:
Also, in Equation (58), the quantity
denotes the cosine of the angle between the particle’s direction and the
-axis, while
denotes the interaction coefficient of the particles with the inner slab’s homogenized material and is defined as follows:
Due to the model’s symmetry, choosing the origin of the coordinate system at the inner slab’s midplane, the appropriate boundary condition for the inner slab is:
The solution of Equations (58) and (61) is the following constant function:
The particle transport equation governing the uncollided angular flux of monoenergetic particles, denoted as
, through the outer slab (“Material B”) in particle directions
has the following form:
where the interaction coefficient
is defined as follows:
The uncollided flux is continuous across the interface at
, i.e.,
For subsequent verification of the expressions to be obtained for the response sensitivities to parameters, the closed-form expression of the uncollided flux
, which is the solution of (63) and (65), is provided below:
The “model response” of interest for this illustrative paradigm model is the total flux of particles interacting with a “kerma response” detector [
11] placed on the outer slab’s external surface, which can be represented mathematically by the following expression:
The interaction coefficient,
, which characterizes the detector’s material, is defined as follows:
where the quantity
denotes the number of distinct nuclides/elements comprising the detector’s response function,
denotes the total microscopic interaction cross section of the respective nuclide with the source particles, and
denotes the respective nuclide’s atomic number density, for
.
Substituting the expression of
obtained in Equation (66) into Equation (67) yields the following expression for the response
:
where the exponential-integral function
is defined as follows:
and possesses the following important properties:
The uncertain parameters that describe the properties (microscopic cross sections, atomic number densities, source, and slab boundaries) characterizing the two materials and the detector will be generically denoted as
, where
denotes the “total number of model parameters.” These parameters will be considered to be the components of the (column) “vector of imprecisely known model parameters”, defined as follows:
From the definitions provided in Equations (59), (60), (64) and (68), it follows that the interaction coefficients and the source are functions of the vector of model parameters.
4.1. Current State-of-the Art Procedure: Application of the 1st-CASAM to Compute First-Order Response Sensitivities Directly with Respect to Model Parameters
The total first-order sensitivity of the response defined in Equation (67) to the model’s primary parameters (including interface and boundary locations) is provided by the Gateaux- (G-) differential
of the response
for arbitrary variations
around the nominal parameter and state functions values, which is defined as follows:
where the “direct-effect” term
depends only on variations
in the parameters, and is defined as follows:
and where the “indirect-effect” term
depends only on variations
in the dependent variable (state function) and is defined as follows:
Since the flux
is known, the direct-effect term in Equation (74) can be computed immediately at this stage, for any parameter variations that appear in its expression. On the contrary, the “indirect-effect” terms in Equation (75) cannot be computed at this stage since the 1st-order variation,
, is not available at this stage. The variational function
could be determined by solving the 1st-Level Variational Sensitivity System (1st-LVSS), which is obtained by G-differentiating Equations (63) and (65) at the nominal parameter values, to obtain the following system of equations:
Carrying out the operations with respect to
in (76) and (77) yields the following 1st-LVSS to be satisfied by the 1st-order forward variational function
:
It is evident that solving Equations (78) and (79) would require at least large-scale computations (i.e., at least one large-scale computation for each parameter variation) to obtain the 1st-order variational function for every parameter variation. Performing so many large-scale computations would be impractical for large-scale systems involving many parameters.
The alternative to solving repeatedly the 1st-LVSS for every possible variation in the system’s imprecisely known parameters is to use the First-Level Adjoint Sensitivity System (1st-LASS), which is constructed by applying the following sequence of steps:
Consider that the function
belongs to a Hilbert space, which will be denoted as
. This Hilbert space is endowed with an inner product of two functions
and
, which is denoted as
and is defined as follows:
Using the definition provided in Equation (80), construct the inner product of a function
with Equation (78) to obtain:
Integrate by parts, over the independent variable
z, the left side of Equation (81) to obtain
Require the last term on the right side of Equation (82) to represent the indirect-effect term defined in Equation (75) by imposing the following relationship:
The relation in Equation (83) implies that the following equation holds, in the weak sense, at the nominal parameter values:
Complete the definition of the function
by requiring that the unknown value of the function
be eliminated from appearing on the right side of Equation (82). This requirement is met by imposing the following condition to be satisfied by the function
on the model’s outer boundary:
Altogether, Equations (84) and (85) constitute a well-posed system, called the “First-Level Adjoint Sensitivity System” (1st-LASS), for determining the function
. The function
is called the “first-level adjoint sensitivity function.” Since the 1st-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain the 1st-level adjoint sensitivity function
For the paradigm model under consideration, the 1st-LASS given by Equations (84) and (85) can be solved exactly to obtain the following closed-form expression for the 1st-level adjoint function
:
where
denotes the Heaviside functional, defined as follows:
Collecting the results obtained in Equation (81) through (85) leads to the following expression for the indirect-effect term:
The appearance of the 1st-level adjoint function in the list of arguments of the indirect-effect term in Equation (88) emphasizes the fact that the function , which depends implicitly on parameter variations, has been eliminated (and has been replaced by the dependence on the first-level adjoint sensitivity function ). The indirect-effect term is now expressed in terms of the 1st-level adjoint function , which does not depend on the model parameter variations. Hence, the 1st-LASS needs to be solved just once (which means a single “large-scale computation” comparable to solving the original equations underlying the model) to obtain the 1st-level adjoint function . After obtaining , all of the partial sensitivities included in the indirect-effect term, cf. Equation (88), are obtained by quadratures for evaluating the integrals involving .
Adding the results obtained in Equations (88) and (74) yields the following complete expression for the total 1st-order response sensitivity
:
The specific expression of each 1st-order partial sensitivity of the response to each uncertain parameter is obtained by identifying the expression that multiplies the respective parameter variation, which yields the following results:
- (i)
Sensitivities stemming solely from the direct-effect term:
- (ii)
Sensitivities stemming solely from the indirect-effect term:
It is noteworthy that the absolute sensitivities
all have equal values, i.e.,
. Also noteworthy are the results indicating that the relative sensitivities of the response with respect to the nuclide densities have the same values as the relative sensitivities of the response with respect to the microscopic cross sections, i.e.,
Notably, obtaining all of the 1st-order partial response sensitivities to the primary model parameters has necessitated a single “large-scale” computation for determining the 1st-level adjoint sensitivity function . This is in contradistinction with the use of the 1st-LVSS, which would require at least large-scale computations to obtain the corresponding first-order sensitivities. Furthermore, the expressions obtained for these sensitivities are exact, as opposed to being approximate, as would have been the case if these sensitivities had been computed by finite-difference or statistical procedures. Also, many more large-scale computations would have been necessary to compute these 1st-order partial response sensitivities by any other (finite-difference or statistical) procedure. Thus, the 1st-CASAM is the most efficient computational method for obtaining the exact expressions of the 1st-order partial response sensitivities to the primary model parameters.
The fact that so many relative sensitivities have the same values, as indicated in Equations (99)–(101), respectively, would render quasi-useless the statistical methods that attempt to rank the sensitivities by their relative values, because such statistical methods break down in situations when many relative sensitivities have equal values. On the other hand, if statistical methods attempted to rank sensitivities by their absolute values, these statistical methods would fail when attempting to determine the response sensitivities to the material properties of the detector. In all cases, the only reliable method for computing response sensitivities accurately (and also most efficiently, from a computational standpoint) is the 1st-CASAM.
4.2. Most Efficient Alternative Procedure: Applying the 1st-FASAM to Compute First-Order Response Sensitivities to Functions/Features of Model Parameters
The particle transport equation governing the uncollided angular flux
, i.e., Equations (63) and (65), can be written in the following form, which highlights the “feature” functions of the primary model parameters:
where:
In terms of the components of the “feature” function
, the uncollided flux and the model response have the following expressions:
The 1st-order total sensitivity of
is obtained by determining the first-order G-differential
of
at the nominal parameter values, which is determined by applying the definition of the G-differential to Equation (104). This operation yields the following expression:
where the direct-effect term
depends only on the parameter variations
and is defined as follows:
and where the indirect-effect term
depends only on the variational function
and is defined as follows:
The 1st-LVSS to be satisfied by the 1st-order forward variational function
is obtained by G-differentiating Equations (102) and (103), to obtain the following system of equations:
It is noteworthy that the above 1st-LVSS would need to be solved just three times to obtain the three variational functions which would correspond to the variations , , and ; thus, would be obtained by setting , , and ; would be obtained by setting , , and ; and would be obtained by setting , , and .
The need for computing the functions
is circumvented by expressing the indirect-effect term
defined in Equation (111) in terms of a 1st-level adjoint function, which will be denoted as
, and which is the solution of a 1st-Level Adjoint Sensitivity System (1st-LASS) constructed by performing the same sequence of operations as indicated in Equations (80)–(82). The resulting 1st-LASS for the 1st-level adjoint function
is the same as was obtained in Equations (84) and (85), namely:
The solution of Equations (114) and (115) has the following expression:
In terms of the 1st-level adjoint function
, the indirect-effect term
will have the following expression:
Adding the expression for the indirect-effect term obtained in Equation (117) with the expression of the direct-effect term obtained in Equation (110) yields the following expression for the total 1st-order G-differential
:
It follows from Equation (118) that the expressions of the partial sensitivities of the response
with respect to the components of the “feature” function
are as follows:
The above expressions can be used to obtain efficiently the sensitivities of the response with respect to the primary model parameters by using the chain-rule of differentiation, as follows:
It can be readily verified that the expressions shown in Equations (122)–(130) are identical to the expressions of the corresponding sensitivities to the model parameters obtained using the 1st-CASAM in Equations (90)–(98).
It is important to note that while there are a total number of first-order sensitivities of the response with respect to the primary model parameters, there are only first-order sensitivities of the response with respect to the five components of the “feature” function , of which three of these sensitivities are with respect to the bona-fide feature functions , while the other two sensitivities are with respect to the boundary parameters , which are “primary parameters” in this illustrative example. The decisive advantages of the 1st-FASAM framework over the 1st-CASAM framework will become apparent when computing the second-order sensitivities, since, as will be shown in the next section, only second-level adjoint sensitivity functions would be needed within the 2nd-FASAM framework, versus the much larger number of second-level adjoint sensitivity functions that would be needed within the 2nd-CASAM framework.
The illustrative paradigm model analyzed in the foregoing part admitted feature functions which depend on mutually exclusive groups of primary parameters, so that none of the components of the “feature” function had any primary parameters in common. It is advantageous to choose components of the “feature” function in this way (i.e., to not depend on any common primary parameters), since it is subsequently easier to compute the sensitivities with respect to the primary parameters from the sensitivities with respect to the components of the “feature” function. Of course, it may not always be possible to choose the components of the “feature” function in order not to share dependencies on the mutually independent primary model parameters. Hence, it may happen that some components of the “feature” function would share dependencies on some common primary parameters. In such instances, particular attention would need to be given to ensuring that the sensitivities of the response with respect to the primary parameters are correctly extracted from the sensitivities of the response with respect to the components of the “feature” function, to ensure that all contributions are properly considered.
5. Introducing the 2nd-FASAM: Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology
The “Second-Order Function/Feature Adjoint Sensitivity Analysis Methodology” (2nd-FASAM) determines the 2nd-order sensitivities of the response with respect to the components of the “feature” function . Conceptually, the first-order sensitivities , which were obtained in Equation (47), are treated as “model responses” and the 2nd-order sensitivities are obtained as the “1st-order sensitivities of the 1st-order sensitivities” by applying the concepts underlying 1st-FASAM to the “response” .
Thus, the first-order G-differential of a first-order sensitivity
,
, is given by the following expression:
where the indirect-effect term
comprises all dependencies on the vectors
and
of variations in the state functions
and
, respectively, and is defined as follows:
The functions
and
are obtained by solving the 2nd-Level Variational Sensitivity System (2nd-LVSS), which consists of concatenating the 1st-LVSS with the G-differentiated 1st-LASS, to obtain the following system (2nd-LVSS):
The argument “2” which appears in the list of arguments of the vector
and of the “variational vector”
in Equation (133) indicates that each of these vectors is a two-block column vector, each block comprising a column vector of dimension
, defined as follows:
To distinguish block vectors from block matrices, two bold capital letters have been used (and will henceforth be used) to denote block matrices, as in the case of “the second-level variational matrix”
. The “2nd-level” is indicated by the superscript “(2)”. The argument “
”, which appears in the list of arguments of
, indicates that this matrix is a
-dimensional block matrix comprising four matrices, each of dimensions
. In particular, the structure of this block matrix is provided below:
where:
The other quantities which appear in Equations (133) and (134) are two-block vectors having the same structure as
, and are defined as follows:
The argument “2” in the expression in Equation (134) indicates that this expression is a two-block column vector comprising two vectors, each of which has -components, all of which are zero-valued.
The need for solving the 2nd-LVSS is circumvented by deriving an alternative expression for the indirect-effect term
defined in Equation (132), in which the function
is replaced by a 2nd-level adjoint function which is independent of variations in the model parameter and state functions. This 2nd-level adjoint function is the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS), which will be constructed by using the same principles as those employed for deriving the 1st-LASS. The 2nd-LASS is constructed in a Hilbert space, denoted as
, which comprises as elements block vectors of the same form as
, and is endowed with the following inner product of two vectors
and
:
The inner product defined in Equation (145) will be used to construct the following 2nd-Level Adjoint Sensitivity System (2nd-LASS) for the 2nd-level adjoint function
, for each
:
subject to boundary conditions represented as follows:
where
The two components
and
of the 2nd-level adjoint function
are distinguished from each other by the use of the numbers “1” and “2”, respectively, in the respective list of arguments. The matrix
comprises
block matrices, each of dimensions
, thus comprising a total of
components (or elements). This matrix is obtained from the following relation:
where the quantity
denotes the corresponding bilinear concomitant with the domain’s boundary, evaluated at the nominal values for the parameters and respective state functions. The 2nd-level adjoint boundary/initial conditions represented by Equation (147) are determined by requiring that:
- (a)
they must be independent of unknown values of ;
- (b)
the substitution of the boundary and/or initial conditions represented by Equations (147) and (134) into the expression of must cause all terms containing unknown values of to vanish.
Using in Equation (132) the relations defining 2nd-LASS together with the 2nd-LVSS and the relation provided in Equation (150) yields the following alternative expression for the indirect-effect term in terms of the 2nd-level adjoint sensitivity function
:
where
denotes residual boundary terms which may not have vanished after having used the boundary and/or initial conditions represented by Equations (134) and (147).
Substituting the expression obtained in Equation (151) into Equation (131) yields the following expression:
where the quantity
denotes the second-order sensitivity of the generic scalar-valued response
with respect to the components
and
of the “feature” function
, computed at the nominal values of the parameters and respective state functions, and has the following expression for
:
Notably, the 2nd-LASS is independent of variations (and, hence, of parameter variations ) and variations in the respective state functions. It is also important to note that the -dimensional matrix is independent of the index . Only the source-term depends on the index . Therefore, the same solver can be used to invert the matrix in order to solve numerically the 2nd-LASS for each -dependent source in order to obtain the corresponding -dependent -dimensional 2nd-level adjoint function . Computationally, it would be most efficient to store, if possible, the inverse matrix , in order to multiply directly the inverse matrix with the corresponding source term for each index , in order to obtain the corresponding -dependent -dimensional 2nd-level adjoint function .
Since the adjoint matrix is block-diagonal, solving the 2nd-LASS is equivalent to solving two 1st-LASSs, with two different source terms. Thus, the “solvers” and the computer program used for solving the 1st-LASS can also be used for solving the 2nd-LASS. The 2nd-LASS was designated as the “second-level” rather than the “second-order” adjoint sensitivity system, since the 2nd-LASS does not involve any explicit 2nd-order G-derivatives of the operators underlying the original system, but involves the inversion of the same operators that need to be inverted for solving the 1st-LASS.
If the 2nd-LASS is solved times, the 2nd-order mixed sensitivities will be computed twice, in two different ways, in terms of two distinct 2nd-level adjoint functions. Consequently, the symmetry property provides an intrinsic (numerical) verification that the 1st-level adjoint function and the components of the 2nd-level adjoint function are computed accurately.
Similarities and Differences between the 2nd-FASAM and the 2nd-CASAM
As has been discussed in
Section 3.1, the “First-Order Comprehensive Adjoint Sensitivity Analysis Methodology” (1st-CASAM) developed by Cacuci [
7,
8] computes the exactly derived expressions of the first-order sensitivities of a model response
directly, with respect to the model’s parameters. In contradistinction, the 1st-FASAM presented in
Section 3 computes the exactly derived expressions of the first-order sensitivities of a model response with respect to functions/features of model. Mathematically, the formal correspondence between the 1st-FASAM and the 1st-CASAM is obtained by replacing the components of the “features” vector-valued function
within the 1st-FASAM formalism with the components of the vector of primary parameters
, which would yield the 1st-CASAM mathematical formalism.
The formal correspondence described above between the 1st-FASAM and the 1st-CASAM also translates, for each of the respective first-order sensitivities, to the formal correspondence between the 2nd-FASAM and the 2nd-CASAM frameworks; i.e., replacing formally the components of in the 2nd-FASAM framework with the components of yields the 2nd-CASAM framework. It is very important to note, however, that while there are a total number of primary model parameters (and, hence, first-order sensitivities) within the 1st-CASAM, there are only feature functions (and, hence, first-order sensitivities), within the 1st-FASAM, where . Each of the respective first-order sensitivity becomes a “response” that generates a corresponding “second-level adjoint sensitivity system” (2nd-LASS) for computing the corresponding second-order sensitivities. Therefore, there are a total number of “second-level adjoint sensitivity systems” to be solved within the 2nd-CASAM framework, but there are only “second-level adjoint sensitivity systems” to be solved within the 2nd-FASAM formalism. Since the 2nd-LASS within either the 2nd-CASAM or the 2nd-CASAM frameworks involves the same operators on their respective left-hand sides (only the “sources” on the respective right-hand sides are different), it follows that the same computational effort is required to solve any of these 2nd-LASS. Thus, since , there are far fewer () large-scale computations needed to solve the 2nd-LASS for obtaining the second-order sensitivities of a response with respect to the components of the “feature” function within the 2nd-FASAM framework, in comparison to the considerably larger number () of large-scale computations needed to solve the 2nd-LASS for obtaining the second-order sensitivities of a response with respect to the components of within the 2nd-CASAM framework.
Performing computations using the 2nd-LASS within the 2nd-CASAM methodology, each such computation corresponding to one of the first-order sensitivities computed within the 1st-CASAM will yield a total of second-order response sensitivities with respect to the primary model parameters , thereby computing the mixed second-order sensitivities twice, using distinct second-level adjoint sensitivity functions. This property of the 2nd-CASAM provides an intrinsic mechanism for verifying the accuracy of the adjoint sensitivity functions produced by solving the corresponding 2nd-LASS.
The above intrinsic verification mechanism, which ensures that the second-level adjoint sensitivity functions are computed accurately, is also inherent to the 2nd-FASAM. Solving the 2nd-LASS corresponding to each of the
first-order sensitivities computed within the 1st-FASAM will yield a total of
second-order response sensitivities with respect to the components of the feature function
, thereby computing the mixed second-order sensitivities twice, using distinct second-level adjoint sensitivity functions. Due to the symmetry of the mixed second-order sensitivities, the two distinct solutions of the respective 2nd-LASS must produce the same numerical result for the second-order sensitivity under consideration. These considerations will be explicitly illustrated in the application to the paradigm particle transport model considered in
Section 6, below.
After applying the 1st-FASAM and the 2nd-FASAM to obtain the first-order and, respectively, second-order sensitivities of a response
with respect to the components of the “feature” function
, the computation of the second-order sensitivities of the response
with respect to the components of the vector
of the primary model parameters is obtained at practically no additional computational costs, by simply using the “chain rule” in conjunction with Equation (57) to obtain:
6. Applying the 2nd-FASAM to Obtain the Second-Order Response Sensitivities to Functions/Features of Parameters for the Paradigm Particle Transport Model Considered in Section 4
This section illustrates the application of the 2nd-FASAM to obtain the exact expressions of the second-order sensitivities of the “kerma” response [
11] with respect to the components of the “feature” function
for the paradigm particle transport model introduced in
Section 4. As has been shown in
Section 5, the 2nd-FASAM obtains the second-order sensitivities by computing “the first-order sensitivities of the first-order sensitivities.” Since the “feature” function
comprises
components, there are five first-order sensitivities
, which were obtained in Equations (119)–(123). The second-order sensitivities
,
of the model response with respect to the components of
will be determined by applying the 2nd-FASAM to the first-order sensitivities
. As has been shown in
Section 5, above, for each first-order sensitivity, there will correspond a “second-level adjoint sensitivity function” which will be used for obtaining the second-order sensitivities that arise from the respective first-order sensitivity. In general, a first-order sensitivity will depend on both the original forward function and on the corresponding adjoint function; in general, therefore, a “second-level adjoint sensitivity function” will be a two-component function, having one component corresponding to the variation in the original forward function and having the second component corresponding to the variation in the first-level adjoint sensitivity function. For the illustrative particle transport model considered in
Section 5, however, an examination of Equations (119)–(123) reveals the following characteristics:
- (i)
The first-order sensitivity depends only on the first-level adjoint sensitivity function . Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from will be obtained in terms of a “two-component second-level adjoint sensitivity function” which will have a single non-zero component, , corresponding to .
- (ii)
The first-order sensitivity is the only first-order sensitivity that depends on both the forward function and on the first-level adjoint sensitivity function . Consequently, when applying the 2nd-FASAM, the second-order sensitivities which will stem from will be obtained in terms of a “two-component second-level adjoint sensitivity function” which will have two non-zero components: one component corresponding to , and the second component corresponding to .
- (iii)
The first-order sensitivity depends only on the forward function, but does not depend on the first-level adjoint sensitivity function . Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from will be obtained in terms of a “two-component second-level adjoint sensitivity function” , which will have a single non-zero component corresponding to .
- (iv)
The first-order sensitivity depends only on the first-level adjoint sensitivity function . Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from will be obtained in terms of a “two-component second-level adjoint sensitivity function” , which will have a single non-zero component corresponding to . This situation is similar to the determination of the second-order sensitivities stemming from , but the respective second-order adjoint sensitivity functions will differ from each other because, although the respective 2nd-Level Adjoint Sensitivity Systems (2nd-LASS) will have the same left-hand side, the respective 2nd-LASS will have distinct right-hand sides (sources).
- (v)
The first-order sensitivity depends only on the forward function ; it does not depend on the first-level adjoint sensitivity function . This situation is similar to the above “case (iii)” for . Hence, when applying the 2nd-FASAM, the second-order sensitivities which will stem from will be obtained in terms of a “two-component second-level adjoint sensitivity function” , which will have a single non-zero component corresponding to . Again, the 2nd-LASS corresponding to and , respectively, will have the same left-hand sides but different right-hand sides (sources).
The above considerations will be detailed in the following five subsections, below.
6.1. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivity
The 1st-order sensitivity
is defined in Equation (119), which is recast into the following “response-like” form:
As has been noted by defining the last term in Equation (155), the 1st-order sensitivity does not depend on the original forward function (i.e., the particle flux), but only depends on the first-level adjoint sensitivity function . The digit “1” in the list of arguments of indicates that is the “first to be considered” of the five first-order sensitivities of the model response with respect to the five components of the “feature” function , which were obtained in Equations (119)–(123).
According to the principles of the 2nd-CASAM presented in
Section 5, above, the second-order sensitivities which stem from
are provided by the first-order G-differential of
. Applying the definition of the G-differential to Equation (155) yields the following relation:
where the direct-effect term
depends only on variations
, while the indirect-effect term
depends only on variations
; these terms are defined, respectively, as follows:
The direct-effect term defined by Equation (157) can be evaluated at this stage by using Equation (116) to obtain:
Inserting the result obtained in Equation (159) into Equation (157) yields:
The indirect-effect term defined in Equation (158) can be evaluated only after having determined the variational function
, which is the solution of the G-differential of the 1st-LASS, which is, in turn, obtained by applying the definition of the (first-order) G-differential to Equations (114) and (115), i.e.:
Carrying out the operations indicated in Equations (161) and (162) yields the following 2nd-Level Forward Variational System (2nd-LVSS) for the function
:
In principle, the 2nd-LVSS could be solved to obtain
for every possible variation in
,
and/or
, which would be prohibitively expensive and also unnecessary. Instead, solving the 2nd-LVSS is avoided altogether by recasting the expression of the indirect-effect term by using a second-level adjoint function, which will be the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS) to be constructed by applying the general 2nd-FASAM as outlined in
Section 5, above. As has been discussed in the foregoing, the indirect-effect term
, defined in Equation (158), only involves the variational function
, but does not also involve the variational function
. Consequently, the 2nd-level adjoint sensitivity function will comprise a single non-zero component (which will correspond to
) rather than two non-zero components—as would have been the case if the first-order sensitivity had depended on both the flux
and the 1st-level adjoint sensitivity function
. Hence, the inner product appropriate for recasting
in terms of a second-level adjoint sensitivity function is the same as that defined in Equation (80). Thus, introducing a one-component 2nd-level adjoint sensitivity function (which is as yet undefined), denoted as
, and forming the inner product of this function with Equation (163), yields the following relation:
Note that the first argument of indicates that only the “2nd-component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in refers to the index/label “,”which indicates that this second-level adjoint sensitivity function will correspond to “the first considered” first-order sensitivity.
Integrating by parts the left side of Equation (165) over the independent variable
z yields the following relation:
The last term on the right side of Equation (166) is now required to represent the indirect-effect term defined in Equation (158) by imposing the following relationship:
The relation in Equation (167) implies that the following equation holds, in the weak sense, at the nominal parameter values:
The definition of the function
is now completed by requiring that the unknown value of the function
be eliminated from appearing on the right side of Equation (168). This requirement is met by imposing the following condition to be satisfied by the function
on the model’s boundary:
Altogether, Equations (168) and (169) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS), for determining the second-level adjoint sensitivity function
. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain
For the paradigm model under consideration, the 1st-LASS given by Equations (168) and (169) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function
:
Collecting the results obtained in Equations (163)–(170) leads to the following expression for the indirect-effect term:
Note that the dependence of on has been replaced in the last equality in Equation (171) by the dependence on .
Adding the results obtained in Equations (171) and (160) yields the following relation:
Note that the dependence of
on the variational function
has been replaced in Equation (172) by the dependence on the second-level adjoint sensitivity function
. Identifying the expressions that multiply the respective components of
, and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (170), yields the following results for the second-order sensitivities that stem from the first-order sensitivity
:
The validity of the expressions obtained in Equations (173)–(177) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).
6.2. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities
The 1st-order sensitivity
is defined in Equation (120), which is recast into the following “response-like” form:
The digit “2” in the list of arguments of indicates that this is the “2nd first-order sensitivity to be considered” of the five first-order sensitivities which were obtained in Equations (119)–(123). As indicated in Equation (178), the first-order sensitivity depends on both the original forward function and also on the first-level adjoint sensitivity function .
According to the principles of the 2nd-CASAM presented in
Section 5, above, the second-order sensitivities which stem from
are provided by the first-order G-differential of
. Applying the definition of the G-differential to Equation (178) yields the following relation:
where the direct-effect term
depends only on variations
, while the indirect-effect term
depends only on variations
and
; these two terms are defined, respectively, as follows:
The direct-effect term defined by Equation (180) can be evaluated at this stage by using Equations (107) and (116) to obtain:
Note that the first term on the right-hand side of Equation (182) vanishes because of Equation (115). The indirect-effect term defined in Equation (181) can be evaluated only after having determined the variational functions and . Recall that the variation is the solution of Equations (163) and (164), while the variation is the solution of Equations (112) and (113). The need for solving repeatedly these equations to obtain the functions and for all possible parameter variation is circumvented by applying the 2nd-FASAM. Since the indirect-effect term depends on both and , the second-level adjoint function needed for recasting the expression of the indirect-effect term will have two non-zero components, and the inner product will therefore have the form shown in Equation (145). Thus, forming the inner product of an as yet undefined second-level function of the form with Equations (112) and (163), respectively, yields the following relation:
Integrating by parts the left side of Equation (183) over the independent variable z yields the following relation:
The second and the last term on the right side of Equation (184) are now required to represent the indirect-effect term defined in Equation (181). This requirement is achieved by imposing the following relationships for
:
The definition of the two-component function
is now completed by requiring that the unknown values of the functions
and
be eliminated from appearing on the right side of Equation (184). This requirement is met by imposing the following conditions to be satisfied by the function
on the model’s boundaries:
Altogether, Equations (185)–(188) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS) for determining the second-level adjoint sensitivity function
. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain
. For the paradigm model under consideration, the 2nd-LASS given by Equations (185)–(188) can be solved exactly to obtain the following closed-form expression for the two non-zero components of the 2nd-level adjoint function
:
Collecting the results obtained in Equations (181)–(188) while using the boundary conditions provided in Equations (113) and (164) leads to the following expression for the indirect-effect term:
Note that the dependence of on the variational functions and has been replaced in Equation (191) by the dependence on the second-level adjoint sensitivity function .
Adding the results obtained in Equations (191) and (182) yields the following relation:
Identifying the expressions that multiply the respective components of
in Equation (192) and using the expressions of the components of the 2nd-level adjoint sensitivity function
yields the following results for the second-order sensitivities that stem from the first-order sensitivity
:
The validity of the expressions obtained in Equations (193)–(197) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).
6.3. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities
The 1st-order sensitivity
is defined in Equation (121), which is recast into the following “response-like” form:
It is evident that the 1st-order sensitivity does not depend on the first-level adjoint sensitivity function , but only depends on the original forward function . The digit “3” in the list of arguments of indicates that this first-order sensitivity is the “third to be considered” of the five first-order sensitivities of the model response (with respect to the five components of the “feature” function ), which were obtained in Equations (119)–(123).
According to the principles of the 2nd-CASAM presented in
Section 5, the second-order sensitivities which stem from
are provided by the first-order G-differential of
. Applying the definition of the G-differential to Equation (198) yields the following relation:
where the direct-effect term
depends only on variations
, while the indirect-effect term
depends only on the variational function
; these terms are defined, respectively, as follows:
The direct-effect term defined by Equation (200) can be evaluated at this stage by using Equation (107) to obtain:
The indirect-effect term defined in Equation (201) can be evaluated after having determined the variational function
, which is the solution of Equations (112) and (113). As before, the need for solving the respective variational equations is avoided by constructing an appropriate 2nd-LASS, by applying the principles of the 2nd-FASAM. This 2nd-LASS is constructed by forming the inner product of Equation (112) with an as yet undefined function
, to obtain the following relation:
Note that the first argument of indicates that only the “ first component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in refers to the index/label “”, which indicates that this second-level adjoint sensitivity function will correspond to “the third considered” first-order sensitivity (namely: ).
Integrating the left side of Equation (203) by parts over the independent variable
z yields the following relation:
The last term on the right side of Equation (204) is now required to represent the indirect-effect term defined in Equation (201), which is achieved by imposing the following relationship over the domain
:
The definition of the function
is now completed by requiring that the unknown value of the function
be eliminated from appearing on the right side of Equation (204). This requirement is met by imposing the following condition to be satisfied by the function
on the model’s outer boundary:
Altogether, Equations (205) and (206) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS), for determining the second-level adjoint sensitivity function
. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain
. For the paradigm model under consideration, the 1st-LASS given by Equations (205) and (206) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function
:
Collecting the results obtained in Equations (201)–(206) leads to the following expression for the indirect-effect term:
Note that the dependence of on the variational function has been replaced in Equation (208) by the dependence on the second-level adjoint sensitivity function .
Adding the results obtained in Equations (208) and (202) yields the following relation:
Note that the dependence of on the variational function has been replaced in Equation (209) by the dependence on .
Identifying the expressions that multiply the respective components of
and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (207) yields the following results for the second-order sensitivities that stem from the first-order sensitivity
:
The validity of the expressions obtained in Equations (210)–(214) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).
6.4. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities
The 1st-order sensitivity
is defined in Equation (122), which is recast into the following “response-like” form:
It is evident that the 1st-order sensitivity does not depend on the original forward function , but only depends on the first-level adjoint sensitivity function . The digit “4” in the list of arguments of indicates that this first-order sensitivity is the “fourth to be considered” of the five first-order sensitivities of the model response with respect to the five components of the “feature” function , which were obtained in Equations (119)–(123).
According to the principles of the 2nd-CASAM presented in
Section 5, the second-order sensitivities which stem from
are provided by the first-order G-differential of
. Applying the definition of the G-differential to Equation (215) yields the following relation:
where the direct-effect term
depends only on variations
, while the indirect-effect term
depends only on variations
; these terms are defined, respectively, as follows:
The direct-effect term defined by Equation (217) can be evaluated at this stage by using Equations (116) and (159) to obtain:
The indirect-effect term defined in Equation (218) can be evaluated after having determined the variational function
, which is the solution of Equations (163) and (164), but the need for solving variational equations is avoided by constructing an appropriate 2nd-LASS, by applying the principles of the 2nd-FASAM. This 2nd-LASS is constructed by forming the inner product of Equation (163) with an as yet undefined function
, to obtain the following relation:
Note that the first argument of indicates that only the “second component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in refers to the index/label “,”which indicates that this second-level adjoint sensitivity function will correspond to “the fourth considered” first-order sensitivity.
Integrating by parts the left side of Equation (220) over the independent variable
z yields the following relation:
The last term on the right side of Equation (221) is now required to represent the indirect-effect term defined in Equation (218) by imposing the following relationship:
The relation in Equation (222) implies that the following equation holds, in the weak sense, at the nominal parameter values for
:
The definition of the function
is now completed by requiring that the unknown value of the function
be eliminated from appearing on the right side of Equation (221). This requirement is met by imposing the following condition to be satisfied by the function
on the model’s boundary:
Altogether, Equations (223) and (224) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS), for determining the second-level adjoint sensitivity function
. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain
For the paradigm model under consideration, the 1st-LASS given by Equations (223) and (224) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function
:
Collecting the results obtained in Equations (220)–(225) leads to the following expression for the indirect-effect term:
Note that the dependence of on the variational function has been replaced in Equation (226) by the dependence on the second-level adjoint sensitivity function .
Adding the results obtained in Equations (226) and (219) (or Equation (217)) yields the following relation:
Identifying the expressions that multiply the respective components of
and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (225) yields the following results for the second-order sensitivities that stem from the first-order sensitivity
:
The validity of the expressions obtained in Equations (228)–(232) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).
6.5. Applying the 2nd-FASAM to Compute the 2nd-Order Response Sensitivities Stemming from the 1st-Order Sensitivities
The 1st-order sensitivity
is defined in Equation (123), which is recast into the following “response-like” form:
As Equation (233) indicates, the 1st-order sensitivity does not depend on the first-level adjoint sensitivity function , but only depends on the original forward function . The digit “5” in the list of arguments of indicates that this first-order sensitivity is the “fifth to be considered” (of the five first-order sensitivities of the model response with respect to the five components of the “feature” function ), which were obtained in Equations (119)–(123).
According to the principles of the 2nd-CASAM presented in
Section 5, the second-order sensitivities which stem from
are provided by the first-order G-differential of
. Applying the definition of the G-differential to Equation (233) yields the following relation:
where the direct-effect term
depends only on variations
, while the indirect-effect term
depends only on variations
; these terms are defined, respectively, as follows:
The direct-effect term defined by Equation (235) can be evaluated at this stage by using Equation (107) to obtain:
The indirect-effect term defined in Equation (236) can be evaluated after having determined the variational function
, which is the solution of Equations (112) and (113), but the need to solve the respective variational equations is avoided by constructing an appropriate 2nd-LASS, by applying the principles of the 2nd-FASAM. This 2nd-LASS is constructed by forming the inner product of Equation (112) with an as yet undefined function
, to obtain the following relation:
Note that the first argument of indicates that only the “first component” of this “two-component second-level adjoint sensitivity function” is non-zero, while the second argument in refers to the index/label “”, which indicates that this second-level adjoint sensitivity function will correspond to “the third considered” first-order sensitivity (namely: ).
Integrating the left side of Equation (238) by parts over the independent variable
z yields the following relation:
The last term on the right side of Equation (239) is now required to represent the indirect-effect term defined in Equation (236), which is achieved by imposing the following relationship over the domain
:
The definition of the function
is now completed by requiring that the unknown value of the function
be eliminated from appearing on the right side of Equation (239). This requirement is fulfilled by imposing the following condition to be satisfied by the function
on the model’s outer boundary:
Altogether, Equations (240) and (241) constitute the “Second-Level Adjoint Sensitivity System” (2nd-LASS) for determining the second-level adjoint sensitivity function
. Since the 2nd-LASS does not depend on the parameter variations, it needs to be solved only once in order to obtain
. For the paradigm model under consideration, the 1st-LASS given by Equations (240) and (241) can be solved exactly to obtain the following closed-form expression for the 2nd-level adjoint function
:
Collecting the results obtained in Equations (236)–(241) leads to the following expression for the indirect-effect term:
Note that the dependence of on the variational function has been replaced in Equation (243) by the dependence on the second-level adjoint sensitivity function .
Adding the results obtained in Equations (243) and (237) yields the following relation:
Identifying the expressions that multiply the respective components of
and using the expression for the 2nd-level adjoint sensitivity function provided in Equation (242) yields the following results for the second-order sensitivities that stem from the first-order sensitivity
:
The validity of the expressions obtained in Equations (245)–(249) can be verified by using the expressions of the first-order sensitivities provided in Equations (119)–(123).
6.6. Comparing Computational Efficiencies: The 2nd-FASAM vs. 2nd-CASAM
As has been shown in the foregoing
Section 6.1–6.5, each of the five first-order sensitivities of the model response to the components of the feature function
has given rise to five second-order sensitivities. The ten mixed second-order sensitivities have been computed twice, each time using two distinct second-level adjoint sensitivity functions, as follows:
As the relations shown in Equations (250)–(259) indicate, the 2nd-FASAM provides an intrinsic verification, based on the symmetries inherent to the mixed second-order sensitivities, of having correctly solved the original forward system, the 1st-LASS, and the five 2nd-LASS for the five second-level adjoint sensitivity functions, all of which are involved in the expressions of the mixed second-order sensitivities.
The determination of the second-order response sensitivities to the primary model parameters
, where
, defined in Equation (72), is accomplished by using the chain rule provided in Equation (154), together with the expressions of the five first-order response sensitivities with respect to the components of the feature function
, as provided in
Section 4.2, Equations (119)–(123), and the second-order response sensitivities with respect to the components of the feature function provided in
Section 6.1,
Section 6.2,
Section 6.3,
Section 6.4 and
Section 6.5.
Alternatively, it would have been possible to apply the 2nd-CASAM [
7,
8] to determine the second-order sensitivities (of the response) directly, with respect to the primary parameters
, by using the
first-order sensitivities with respect to the primary parameters. Evidently, following this procedure would have required
large-scale computations for solving the TP 2nd-LASSystems, each of which would have corresponded to one of the
first-order sensitivities. It is evident that two of these computations would stem from the first-order sensitivities to the boundary parameters
and
, which are also responsible for two of the five 2nd-LASS computations involved in the 2nd-FASAM; recall that the “feature” function
comprises just three bona fide “features”, namely
,
and
.
Even for the simple paradigm shielding model considered in this work, it is evident that the 2nd-FASAM is computationally significantly more advantageous to use rather than the 2nd-CASAM: the 2nd-FASAM requires five large-scale computations (plus insignificant additional analytic computations for applying the chain rule) to obtaining all of the second-order sensitivities with respect to the primary model parameters, while the 2nd-CASAM would require large-scale computations to obtain these second-order sensitivities. By comparison, even the simplest-minded four-point finite-difference scheme would require large-scale computations, and would at best provide approximate values (whin second-order errors of the finite-difference step-size) for the respective second-order sensitivities.