*3.1. Covariance*

Let the continuous variables *a* and *b* have chaos expansions

$$a(\mathbf{x}, \boldsymbol{\xi}) = \sum\_{j=0}^{\infty} a\_j(\mathbf{x}) \Psi\_j(\boldsymbol{\xi}) \qquad \text{and} \qquad \qquad b(\boldsymbol{z}, \boldsymbol{\zeta}) = \sum\_{k=0}^{\infty} \beta\_k(\boldsymbol{z}) \Phi\_k(\boldsymbol{\zeta}) \,. \tag{11}$$

The covariance between *a* and *b* can be expressed in terms of two nested expected values

$$\text{cov}(a, b) = \text{E}[(a - \text{E}[a])(b - \text{E}[b])]\,,$$

the external of which can be expressed as a double integral yielding

$$\text{cov}(a, b) = \int\_{\mathcal{A}} \int\_{\mathbb{B}} (a - \mathcal{E}[a])(b - \mathcal{E}[b]) db da \,, \tag{12}$$

where A and B are the supports of *a* and *b* respectively. Substituting the expansions from Equation (11) into Equation (12) and acknowledging that the zeroth coefficient is the expected value gives

$$\begin{split} \text{cov}(a,b) &= \int\_{\mathcal{A}} \int\_{\mathbb{B}} (ab - a\mathbb{E}[b] - b\mathbb{E}[a] + \mathbb{E}[a]\mathbb{E}[b]) db da \\ &= \int\_{\mathcal{A}} \int\_{\mathbb{B}} (ab - \beta\_0 a - a\_0 b + a\_0 \beta\_0) db da \\ &= -a\_0 \beta\_0 + \int\_{\mathcal{A}} \int\_{\mathbb{B}} (ab) db da \end{split} \tag{13a}$$

$$=-\mathfrak{a}\rho\mathfrak{\beta}\_{0} + \int\_{\mathcal{X}} \overline{\int}\_{\mathcal{Z}} \sum\_{j=0}^{\infty} a\_{j}(\mathfrak{x}) \Psi\_{j}(\mathfrak{f}) \sum\_{k=0}^{\infty} \beta\_{k}(\mathfrak{z}) \Phi\_{k}(\mathfrak{f}) d\mathfrak{f} d\mathfrak{f} \,. \tag{13b}$$

Note the change of variables between Equations (13a) and (13b). This is possible because the random variable and the weight function (*a*/*ξ* and *b*/*ζ* in this case) are over the same support. Additionally, the notation of the support variable is changed to be consistent with the integration variable.

As long as the covariance is finite, the summation and the integrals can be interchanged [49], giving a final generalized expression for the covariance to be

$$\text{cov}(a, b) = \sum\_{j=1}^{\infty} \sum\_{k=1}^{\infty} a\_j(\mathbf{x}) \beta\_k(\mathbf{z}) \int\_{\mathcal{X}} \int\_{\mathcal{Z}} \Psi\_j(\boldsymbol{\xi}) \Phi\_k(\boldsymbol{\zeta}) d\boldsymbol{\xi} d\boldsymbol{\xi} \,. \tag{14}$$

In general, no further simplifications can be made; however, if the variables *x* and *z* are expanded using the same set of basis polynomials, then integration reduces to

$$\text{cov}(a, b) = \sum\_{k=1}^{\infty} \sum\_{j=1}^{\infty} a\_k(\mathbf{x}) \beta\_j(z) \int\_{\mathcal{X}} \Psi\_k(\xi) \Psi\_j(\xi) p(\xi) d\xi \,\,\,\tag{15}$$

containing a single variable with respect to the base pdf. Taking advantage of the basis polynomial orthogonality yields the following simple expression:

$$\text{cov}(a, b) = \sum\_{k=1}^{\infty} a\_k(\mathbf{x}) \beta\_k(\mathbf{z}) \langle \Psi\_k^2 \rangle\_{p(\vec{\xi})} \,. \tag{16}$$

Combined with the variance, the covariance matrix of the 2 × 2 system of *x* and *z* just discussed is given as

$$P = \sum\_{k=1}^{\infty} \begin{bmatrix} \alpha\_k^2 & \alpha\_k \beta\_k \\ \alpha\_k \beta\_k & \beta\_k^2 \end{bmatrix} \langle \Psi\_k^2 \rangle\_{p(\vec{\xi})} \dots$$

For an *n*-dimensional state, let be the *n* × ∞ matrix for the *n*, theoretically infinite, chaos coefficients. Written generally, the covariance matrix in terms of a chaos expansion is

$$P = \sum\_{k=1}^{\infty} \mathbf{e}\_k \mathbf{e}\_k^T \langle \Psi\_k^2 \rangle\_{p(\xi)} \cdot$$

In cases where orthonormal polynomials are used, the polynomial inner product disappears completely leaving only the summation of the estimated chaos coefficients

$$P = \sum\_{k=1}^{\infty} \mathfrak{e}\_k \mathfrak{e}\_k^T \,. \tag{17}$$
