Exact Expressions for Kullback–Leibler Divergence for Multivariate and Matrix-Variate Distributions

Abstract

The Kullback–Leibler divergence is a measure of the divergence between two probability distributions, often used in statistics and information theory. However, exact expressions for it are not known for multivariate or matrix-variate distributions apart from a few cases. In this paper, exact expressions for the Kullback–Leibler divergence are derived for over twenty multivariate and matrix-variate distributions. The expressions involve various special functions.

1. Introduction

The Kullback–Leibler divergence (KLD) due to [1] is a fundamental concept in information theory and statistics used to measure the divergence between two probability distributions. It quantifies how one probability distribution diverges from a second, reference probability distribution. Specifically, it calculates the expected extra amount of information required to represent data sampled from one distribution using a code optimized for another distribution. The KLD is asymmetric and not a true metric as it does not satisfy the triangle inequality. It is widely employed in various fields including machine learning, where it serves as a key component in tasks such as model comparison, optimization, and generative modeling, providing a measure of dissimilarity or discrepancy between probability distributions [2].

Suppose $\mathbf{X}$ is a continuous vector-variate random variable or a continuous matrix-variate random variable having one of two probability density functions $f_i\left(·;{\mathit{\theta }}_i\right)$, $i=1,2$ parameterized by ${\mathit{\theta }}_i$, $i=1,2$. The KLD between $f_1\left(·;{\mathit{\theta }}_1\right)$ and $f_2\left(·;{\mathit{\theta }}_2\right)$ is defined by

$$\begin{array}{c}{\displaystyle KLD=E\left[log\frac{f_1\left(\mathbf{X};{\mathit{\theta }}_1\right)}{f_2\left(\mathbf{X};{\mathit{\theta }}_2\right)}\right],}\hfill \end{array}$$

(1)

where the expectation is with respect to $f_1\left(·;{\mathit{\theta }}_1\right)$.

Because of the increasing applications of the KLD, it is useful to have exact expressions for (1). Apart from the multivariate normal distribution, not many expressions have been derived for (1) for multivariate or matrix-variate distributions. The KLD for the multivariate generalized Gaussian distribution was derived only in 2019, see [3]. The KLD for the multivariate Cauchy distribution was derived only in 2022, see [4]. The KLD for the multivariate t distribution was derived only in 2023, see [5].

The aim of this paper is to derive exact expressions for (1) for over twenty multivariate and matrix-variate distributions. The exact expressions for multivariate distributions are presented in Section 2. The exact expressions for matrix-variate distributions are presented in Section 3. The derivations of all of the expressions including a technical lemma needed for the derivations are presented in Section 4. The distributions considered in this paper are continuous. We shall not be considering discrete distributions including mixtures.

The functions and parameters used in this paper are all real-valued. The calculations involve several real-valued special functions listed in the Appendix A.

2. Exact Expressions for Multivariate Distributions

In this section, we state the exact expressions for (1) for Dirichlet, multivariate generalized Gaussian, inverted Dirichlet, multivariate Gauss hypergeometric, multivariate Kotz type, ref. [6]’s multivariate logistic, ref. [7]’s multivariate logistic, ref. [8]’s multivariate normal, multivariate Pearson type II, multivariate Selberg beta, multivariate weighted exponential and von Mises distributions.

A closed form for (1) for the multivariate generalized Gaussian distribution was derived by [3]. But it involved a special function defined as a $(p-1)$ folded infinite sum. The expression we give in Section 2.2 is much simpler in that it involves a single infinite sum. A closed form for (1) for the Dirichlet distribution is available in [9] and https://statproofbook.github.io/P/dir-kl.html (accessed on 1 July 2024).

2.1. Dirichlet Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\displaystyle \prod _{i=1}^K{x}_i^{a_i-1}}}{{\displaystyle B\left(a_1,\dots ,a_{K-1};a_K\right)}}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\displaystyle \prod _{i=1}^K{x}_i^{b_i-1}}}{{\displaystyle B\left(b_1,\dots ,b_{K-1};b_K\right)}}}\hfill \end{array}$$

for $K\ge 2$, $a_1>0,\dots ,a_K>0$, $b_1>0,\dots ,b_K>0$, $0\le x_1\le 1,\dots ,0\le x_K\le 1$ and $x_1+\cdots +x_K=1$. The corresponding KLD is

$$\begin{array}{c}{\displaystyle KLD=log\left[\frac{B\left(b_1,\dots ,b_{K-1};b_K\right)}{B\left(a_1,\dots ,a_{K-1};a_K\right)}\right]+\sum _{i=1}^K\left(a_i-b_i\right)\psi \left(a_i\right)-\psi \left(a_1+\cdots +a_K\right)\sum _{i=1}^K\left(a_i-b_i\right).}\hfill \end{array}$$

2.2. Multivariate Generalized Gaussian Distribution ([10], p. 215)

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(x_1,\dots ,x_p\right)=\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\left|{\mathbf{V}}_1\right|}^{-\frac{1}{2}}exp\left\{-{\left({\mathbf{x}}^T{\mathbf{V}}_1^{-1}\mathbf{x}\right)}^{\frac{\alpha }{2}}\right\}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(x_1,\dots ,x_p\right)=\frac{\beta \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\beta }\right)}{\left|{\mathbf{V}}_2\right|}^{-\frac{1}{2}}exp\left\{-{\left({\mathbf{x}}^T{\mathbf{V}}_2^{-1}\mathbf{x}\right)}^{\frac{\beta }{2}}\right\}}\hfill \end{array}$$

for $-\infty <x_1<\infty ,\dots ,-\infty <x_p<\infty$, $\alpha >0$, $\beta >0$ and ${\mathbf{V}}_1$, ${\mathbf{V}}_2$ positive definite symmetric matrices. The corresponding KLD is

$$\begin{array}{cc}KLD=\hfill & log\left[\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{\beta }\right)}{\beta \mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}\right]+\frac{1}{2}log\frac{\left|{\mathbf{V}}_2\right|}{\left|{\mathbf{V}}_1\right|}-\frac{p}{\alpha }+\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{p+\beta }{2}\right)}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}\sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left(\genfrac{}{}{0pt}{}{\frac{\beta }{2}}{r_1}\right)\\ & {\displaystyle \times \left(\genfrac{}{}{0pt}{}{r_1}{r_2}\right)\cdots \left(\genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}\right){\lambda }_1^{\frac{\beta }{2}-r_1}\prod _{j=1}^{p-1}\left[{\left({\lambda }_{j+1}-{\lambda }_j\right)}^{r_j-r_{j+1}}B\left(r_j+\frac{p-j}{2},\frac{1}{2}\right)\right]}\hfill \end{array}$$

provided that the infinite sum converges.

2.3. Inverted Dirichlet Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\displaystyle \mathsf{\Gamma }\left(a_1+\cdots +a_{K+1}\right)}}{{\displaystyle \mathsf{\Gamma }\left(a_1\right)\cdots \mathsf{\Gamma }\left(a_{K+1}\right)}}{\left(1+\sum _{i=1}^K{x}_i\right)}^{-a_1-\cdots -a_{K+1}}\prod _{i=1}^K{x}_i^{a_i-1}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\displaystyle \mathsf{\Gamma }\left(b_1+\cdots +b_{K+1}\right)}}{{\displaystyle \mathsf{\Gamma }\left(b_1\right)\cdots \mathsf{\Gamma }\left(b_{K+1}\right)}}{\left(1+\sum _{i=1}^K{x}_i\right)}^{-b_1-\cdots -b_{K+1}}\prod _{i=1}^K{x}_i^{b_i-1}}\hfill \end{array}$$

for $K\ge 2$, $a_1>0,\dots ,a_{K+1}>0$, $b_1>0,\dots ,b_{K+1}>0$ and $x_1>0,\dots ,x_K>0$. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\displaystyle \mathsf{\Gamma }\left(a_1+\cdots +a_{K+1}\right)\mathsf{\Gamma }\left(b_1\right)\cdots \mathsf{\Gamma }\left(b_{K+1}\right)}}{{\displaystyle \mathsf{\Gamma }\left(b_1+\cdots +b_{K+1}\right)\mathsf{\Gamma }\left(a_1\right)\cdots \mathsf{\Gamma }\left(a_{K+1}\right)}}\right]+\sum _{i=1}^K\left(a_i-b_i\right)\psi \left(a_i\right)}\hfill \\ \hfill & {\displaystyle -\psi \left(a_{K+1}\right)\sum _{i=1}^K\left(a_i-b_i\right)}\hfill \\ \hfill & {\displaystyle +\left(b_1+\cdots +b_{K+1}-a_1-\cdots -a_{K+1}\right)\left[\psi \left(a_1+\cdots +a_{K+1}\right)-\psi \left(a_{K+1}\right)\right].}\hfill \end{array}$$

2.4. Multivariate Gauss Hypergeometric Distribution [11]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=C\left(a_1,\dots ,a_K,b,c\right)\frac{{\displaystyle {\left(1-\sum _{i=1}^K{x}_i\right)}^{b-1}\prod _{i=1}^K{x}_i^{a_i-1}}}{{\displaystyle {\left(1+\sum _{i=1}^K{x}_i\right)}^c}}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=C\left(d_1,\dots ,d_K,e,f\right)\frac{{\displaystyle {\left(1-\sum _{i=1}^K{x}_i\right)}^{e-1}\prod _{i=1}^K{x}_i^{d_i-1}}}{{\displaystyle {\left(1+\sum _{i=1}^K{x}_i\right)}^f}}}\hfill \end{array}$$

for $K\ge 2$, $a_1>0,\dots ,a_K>0$, $b>0$, $-\infty <c<\infty$, $d_1>0,\dots ,d_K>0$, $e>0$, $-\infty <f<\infty$, $0\le x_1\le 1,\dots ,0\le x_K\le 1$ and $x_1+\cdots +x_K\le 1$. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{C\left(a_1,\dots ,a_K,b,c\right)}{C\left(d_1,\dots ,d_K,e,f\right)}\right]+\sum _{i=1}^K\left(a_i-d_i\right){\left.\frac{\partial }{\partial \alpha }\left[\frac{C\left(a_1,\dots ,a_i,\dots ,a_K,b,c\right)}{C\left(a_1,\dots ,a_i+\alpha ,\dots ,a_K,b,c\right)}\right]\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(b-e){\left.\frac{\partial }{\partial \alpha }\left[\frac{C\left(a_1,\dots ,a_i,\dots ,a_K,b,c\right)}{C\left(a_1,\dots ,a_K,b+\alpha ,c\right)}\right]\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle -(c-f){\left.\frac{\partial }{\partial \alpha }\left[\frac{C\left(a_1,\dots ,a_i,\dots ,a_K,b,c\right)}{C\left(a_1,\dots ,a_K,b,c-\alpha \right)}\right]\right|}_{\alpha =0}.}\hfill \end{array}$$

2.5. Multivariate Kotz Type Distribution [12]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(x_1,\dots ,x_p\right)=\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\left|{\mathbf{\Sigma }}_1\right|}^{-\frac{1}{2}}{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^{N-1}exp\left\{-q{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^a\right\}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(x_1,\dots ,x_p\right)=\frac{b\mathsf{\Gamma }\left(\frac{p}{2}\right)s^{\frac{2M+p-2}{2b}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2M+p-2}{2b}\right)}{\left|{\mathbf{\Sigma }}_2\right|}^{-\frac{1}{2}}{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{x}\right)}^{M-1}exp\left\{-s{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{x}\right)}^b\right\}}\hfill \end{array}$$

for $-\infty <x_1<\infty ,\dots ,-\infty <x_p<\infty$, $a>0$, $q>0$, $N>1-{\displaystyle \frac{p}{2}}$, $b>0$, $s>0$, $M>1-{\displaystyle \frac{p}{2}}$ and ${\mathbf{\Sigma }}_1$, ${\mathbf{\Sigma }}_2$ positive definite symmetric matrices. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{a{q}^{\frac{2N+p-2}{2a}}\mathsf{\Gamma }\left(\frac{2M+p-2}{2b}\right)}{b{s}^{\frac{2M+p-2}{2b}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\right]+\frac{1}{2}log\frac{\left|{\mathbf{\Sigma }}_2\right|}{\left|{\mathbf{\Sigma }}_1\right|}+\frac{N-1}{a}\left[\psi \left(\frac{2N+p-2}{2a}\right)-logq\right]}\hfill \\ \hfill & {\displaystyle -\frac{2N+p-2}{2a}-(M-1)\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{a{\pi }^{\frac{p}{2}}}\left\{\psi \left(\frac{p+2N-2}{2a}\right)-logq\right\}\left[\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)\right]}\hfill \\ \hfill & {\displaystyle -(M-1)\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}log{\lambda }_1\left[\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)\right]}\hfill \\ \hfill & {\displaystyle +(M-1)\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_{p-1}=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_{p-1}}\right)}\hfill \\ \hfill & {\displaystyle \times \prod _{j=1}^{p-1}\left\{{\left[\frac{\left({\lambda }_{j+1}-{\lambda }_j\right)}{{\lambda }_1}\right]}^{i_j}B\left(\sum _{\ell =j}^{p-1}{i}_{\ell }+\frac{p-j}{2},\frac{1}{2}\right)\right\}}\hfill \\ \hfill & {\displaystyle +\frac{s\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{2N+p+2b-2}{2a}\right)}{{\pi }^{\frac{p}{2}}{q}^{\frac{b}{a}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left(\genfrac{}{}{0pt}{}{b}{r_1}\right)\left(\genfrac{}{}{0pt}{}{r_1}{r_2}\right)\cdots \left(\genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}\right){\lambda }_1^{b-r_1}}\hfill \\ \hfill & {\displaystyle \times \prod _{j=1}^{p-1}\left[{\left({\lambda }_{j+1}-{\lambda }_j\right)}^{r_j-r_{j+1}}B\left(r_j+\frac{p-j}{2},\frac{1}{2}\right)\right]}\hfill \end{array}$$

provided that the infinite sum converges.

2.6. Multivariate Logistic Distribution [6]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(x_1,\dots ,x_p\right)=\frac{p!a_1\cdots a_{p}exp\left(-a_1{x}_1-\cdots -a_p{x}_p\right)}{{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{p+1}}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(x_1,\dots ,x_p\right)=\frac{p!b_1\cdots b_{p}exp\left(-b_1{x}_1-\cdots -b_p{x}_p\right)}{{\left[1+exp\left(-b_1{x}_1\right)+\cdots +exp\left(-b_p{x}_p\right)\right]}^{p+1}}}\hfill \end{array}$$

for $-\infty <x_1<\infty ,\dots ,-\infty <x_p<\infty$, $a_1>0,\dots ,a_p>0$ and $b_1>0,\dots ,b_p>0$. The corresponding KLD is

$$\begin{array}{cc}KLD\hfill & {\displaystyle =log\left(\frac{a_1\cdots a_p}{b_1\cdots b_p}\right)}\hfill \\ \hfill & {\displaystyle -(p+1)\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_p=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_p}\right)\left[\prod _{j=1}^p\mathsf{\Gamma }\left(\frac{a_j+i_j{b}_j}{a_j}\right)\right]\mathsf{\Gamma }\left(1-\sum _{j=1}^p\frac{i_j{b}_j}{a_j}\right)}\hfill \\ \hfill & {\displaystyle -(p+1)\frac{{\mathsf{\Gamma }}^{\prime }(p+1)-\mathsf{\Gamma }(p+1){\mathsf{\Gamma }}^{\prime }\left(1\right)}{p!}}\hfill \end{array}$$

provided that ${\sum }_{j=1}^p{\displaystyle \frac{i_j{b}_j}{a_j}}<1$ and the infinite series converges.

2.7. Multivariate Logistic Distribution [7]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(x_1,\dots ,x_p\right)=\frac{{\left(b\right)}_p{a}_1\cdots a_{p}exp\left(-a_1{x}_1-\cdots -a_p{x}_p\right)}{{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{b+p}}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(x_1,\dots ,x_p\right)=\frac{{\left(d\right)}_p{c}_1\cdots c_{p}exp\left(-c_1{x}_1-\cdots -c_p{x}_p\right)}{{\left[1+exp\left(-c_1{x}_1\right)+\cdots +exp\left(-c_p{x}_p\right)\right]}^{d+p}}}\hfill \end{array}$$

for $-\infty <x_1<\infty ,\dots ,-\infty <x_p<\infty$, $a_1>0,\dots ,a_p>0$, $c_1>0,\dots ,c_p>0$, $b>0$ and $d>0$. The corresponding KLD is

$$\begin{array}{cc}{\displaystyle KLD}\hfill & {\displaystyle =log\left[\frac{{\left(b\right)}_p{a}_1\cdots a_p}{{\left(d\right)}_p{b}_1\cdots b_p}\right]}\hfill \\ \hfill & {\displaystyle -\frac{d+p}{\mathsf{\Gamma }\left(b\right)}\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_p=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_p}\right)\left[\prod _{j=1}^p\mathsf{\Gamma }\left(\frac{a_j+i_j{c}_j}{a_j}\right)\right]\mathsf{\Gamma }\left(b-\sum _{j=1}^p\frac{i_j{c}_j}{a_j}\right)}\hfill \\ \hfill & {\displaystyle -(b+p)\frac{\mathsf{\Gamma }\left(b\right){\mathsf{\Gamma }}^{\prime }(b+p)-\mathsf{\Gamma }(b+p){\mathsf{\Gamma }}^{\prime }\left(b\right)}{\mathsf{\Gamma }\left(b\right)\mathsf{\Gamma }(b+p)}}\hfill \end{array}$$

provided that ${\sum }_{j=1}^p{\displaystyle \frac{i_j{c}_j}{a_j}}<b$ and the infinite series converges.

2.8. Sarabia [8]’s Multivariate Normal Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(x_1,\dots ,x_p\right)=\frac{\sqrt{a_1\cdots a_p}{\beta }_p\left(c,a_1,\dots ,a_p\right)}{{\left(2\pi \right)}^{\frac{p}{2}}}exp\left\{-\frac{1}{2}\left[\sum _{i=1}^p\left(a_i{x}_i^2\right)+c\prod _{i=1}^p\left(a_i{x}_i^2\right)\right]\right\}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(x_1,\dots ,x_p\right)=\frac{\sqrt{b_1\cdots b_p}{\beta }_p\left(d,b_1,\dots ,b_p\right)}{{\left(2\pi \right)}^{\frac{p}{2}}}exp\left\{-\frac{1}{2}\left[\sum _{i=1}^p\left(b_i{x}_i^2\right)+d\prod _{i=1}^p\left(b_i{x}_i^2\right)\right]\right\}}\hfill \end{array}$$

for $-\infty <x_1<\infty ,\dots ,-\infty <x_p<\infty$, $a_1>0,\dots ,a_p>0$, $b_1>0,\dots ,b_p>0$, $c>0$ and $d>0$, where ${\beta }_p\left(c,a_1,\dots ,a_p\right)$ and ${\beta }_p\left(d,b_1,\dots ,b_p\right)$ denote normalizing constants. The corresponding KLD is

$$\begin{array}{cc}KLD\hfill & {\displaystyle =log\left[\frac{\sqrt{a_1\cdots a_p}{\beta }_p\left(c,a_1,\dots ,a_p\right)}{\sqrt{b_1\cdots b_p}{\beta }_p\left(d,b_1,\dots ,b_p\right)}\right]}\hfill \\ \hfill & {\displaystyle +\left[\frac{1}{2}-\frac{c{\beta }_p^{\prime }\left(c,a_1,\dots ,a_p\right)}{{\beta }_p\left(c,a_1,\dots ,a_p\right)}\right]\sum _{i=1}^p\left(\frac{b_i}{a_i}-1\right)}\hfill \\ \hfill & {\displaystyle +\left(\frac{d{b}_1\cdots b_p}{a_1\cdots a_p}-c\right)\frac{{\beta }_p^{\prime }\left(c,a_1,\dots ,a_p\right)}{{\beta }_p\left(c,a_1,\dots ,a_p\right)},}\hfill \end{array}$$

where ${\beta }_p^{\prime }\left(c,a_1,\dots ,a_p\right)={\displaystyle \frac{\partial }{\partial c}}{\beta }_p\left(c,a_1,\dots ,a_p\right)$.

2.9. Multivariate Pearson Type II Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\pi }^{-\frac{K}{2}}\mathsf{\Gamma }\left(\frac{K}{2}\right)\mathsf{\Gamma }\left(\frac{K}{2}+a+b-1\right)}{\mathsf{\Gamma }\left(a\right)\mathsf{\Gamma }\left(\frac{K}{2}+b-1\right)}{\left(\sum _{i=1}^K{x}_i^2\right)}^{b-1}{\left(1-\sum _{i=1}^K{x}_i^2\right)}^{a-1}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\pi }^{-\frac{K}{2}}\mathsf{\Gamma }\left(\frac{K}{2}\right)\mathsf{\Gamma }\left(\frac{K}{2}+c+d-1\right)}{\mathsf{\Gamma }\left(c\right)\mathsf{\Gamma }\left(\frac{K}{2}+d-1\right)}{\left(\sum _{i=1}^K{x}_i^2\right)}^{d-1}{\left(1-\sum _{i=1}^K{x}_i^2\right)}^{c-1}}\hfill \end{array}$$

for $K\ge 2$, $a>0$, $b>0$, $c>0$, $d>0$ and $0<x_1^2+\cdots +x_K^2<1$. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{\mathsf{\Gamma }\left(\frac{K}{2}+a+b-1\right)\mathsf{\Gamma }\left(c\right)\mathsf{\Gamma }\left(\frac{K}{2}+d-1\right)}{\mathsf{\Gamma }\left(a\right)\mathsf{\Gamma }\left(\frac{K}{2}+b-1\right)\mathsf{\Gamma }\left(\frac{K}{2}+c+d-1\right)}\right]}\hfill \\ \hfill & {\displaystyle +(b-d)\left[\psi \left(\frac{K}{2}+b-1\right)-\psi \left(\frac{K}{2}+a+b-1\right)\right]}\hfill \\ \hfill & {\displaystyle +(a-c)\left[\psi \left(a\right)-\psi \left(\frac{K}{2}+a+b-1\right)\right].}\hfill \end{array}$$

2.10. Multivariate Selberg Beta Distribution [13]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=C\left(a,b,c\right){\left|\prod _{1\le i<j\le p}\left(x_i-x_j\right)\right|}^{2c}\prod _{i=1}^p\left[x_i^{a-1}{\left(1-x_i\right)}^{b-1}\right]}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=C\left(d,e,f\right){\left|\prod _{1\le i<j\le p}\left(x_i-x_j\right)\right|}^{2f}\prod _{i=1}^p\left[x_i^{d-1}{\left(1-x_i\right)}^{e-1}\right]}\hfill \end{array}$$

for $a>0$, $b>0$, $c>0$, $d>0$, $e>0$, $f>0$ and $0<x_1<1,\dots ,0<x_p<1$. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{C\left(d,e,f\right)}{C\left(a,b,c\right)}\right]+(a-d){\left.\frac{\partial }{\partial \alpha }\left[\frac{C(a,b,c)}{C(a+\alpha ,b,c)}\right]\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(b-e){\left.\frac{\partial }{\partial \alpha }\left[\frac{C(a,b,c)}{C(a,b+\alpha ,c)}\right]\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +2(c-f){\left.\frac{\partial }{\partial \alpha }\left[\frac{C(a,b,c)}{C(a,b,c+\alpha )}\right]\right|}_{\alpha =0}.}\hfill \end{array}$$

2.11. Multivariate Weighted Exponential Distribution [14]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(x_1,\dots ,x_p\right)=\frac{{\displaystyle \left(\sum _{i=1}^p{a}_i\right)\left(\prod _{i=1}^p{a}_i\right)}}{{\displaystyle a_{p+1}}}\left\{1-exp\left[-a_{p+1}min\left(x_1,\dots ,x_p\right)\right]\right\}\left[\prod _{i=1}^{p}exp\left(-a_i{x}_i\right)\right]}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(x_1,\dots ,x_p\right)=\frac{{\displaystyle \left(\sum _{i=1}^p{b}_i\right)\left(\prod _{i=1}^p{b}_i\right)}}{{\displaystyle b_{p+1}}}\left\{1-exp\left[-b_{p+1}min\left(x_1,\dots ,x_p\right)\right]\right\}\left[\prod _{i=1}^{p}exp\left(-b_i{x}_i\right)\right]}\hfill \end{array}$$

for $x_1>0,\dots ,x_p>0$, $a_1>0,\dots ,a_{p+1}>0$ and $b_1>0,\dots ,b_{p+1}>0$. The corresponding KLD is

$$\begin{array}{cc}{\displaystyle KLD}\hfill & {\displaystyle =log\left[\frac{{\displaystyle b_{p+1}\left(\sum _{i=1}^p{a}_i\right)\left(\prod _{i=1}^p{a}_i\right)}}{{\displaystyle a_{p+1}\left(\sum _{i=1}^p{b}_i\right)\left(\prod _{i=1}^p{b}_i\right)}}\right]+\sum _{i=1}^p\left(b_i-a_i\right)\left(\frac{1}{a_i}+\frac{1}{a_{p+1}}\right)}\hfill \\ \hfill & {\displaystyle -\sum _{k=1}^{\infty }\frac{\left(a_1+\cdots +a_p\right)\left(a_1+\cdots +a_p+a_{p+1}\right)}{k\left(a_1+\cdots +a_p+k{a}_{p+1}\right)\left[a_1+\cdots +a_p+(k+1)a_{p+1}\right]}}\hfill \\ \hfill & {\displaystyle +\sum _{k=1}^{\infty }\frac{\left(a_1+\cdots +a_p\right)\left(a_1+\cdots +a_p+a_{p+1}\right)}{k\left(a_1+\cdots +a_p+k{b}_{p+1}\right)\left[a_1+\cdots +a_p+a_{p+1}+k{b}_{p+1}\right]},}\hfill \end{array}$$

which follows from properties stated in [14] provided that the infinite series converge.

2.12. Von Mises Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\kappa }_1^{\frac{p}{2}-1}exp\left({\kappa }_1{\mathit{\mu }}_1^T\mathbf{x}\right)}{{\left(2\pi \right)}^{\frac{p}{2}}{I}_{\frac{p}{2}-1}\left({\kappa }_1\right)}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\kappa }_2^{\frac{p}{2}-1}exp\left({\kappa }_2{\mathit{\mu }}_2^T\mathbf{x}\right)}{{\left(2\pi \right)}^{\frac{p}{2}}{I}_{\frac{p}{2}-1}\left({\kappa }_2\right)}}\hfill \end{array}$$

for ${\kappa }_1>0$, ${\kappa }_2>0$, ${\mathit{\mu }}_1^T{\mathit{\mu }}_1=1$, ${\mathit{\mu }}_2^T{\mathit{\mu }}_2=1$ and ${\mathbf{x}}^T\mathbf{x}=1$. The corresponding KLD is

$$\begin{array}{c}{\displaystyle KLD=log\left[\frac{{\kappa }_1^{\frac{p}{2}-1}}{{\kappa }_2^{\frac{p}{2}-1}}\frac{I_{\frac{p}{2}-1}\left({\kappa }_2\right)}{I_{\frac{p}{2}-1}\left({\kappa }_1\right)}\right]+\left({\kappa }_1-{\kappa }_2{\mathit{\mu }}_2^T{\mathit{\mu }}_1\right).}\hfill \end{array}$$

3. Exact Expressions for Matrix-Variate Distributions

In this section, we state exact expressions for (1) for matrix-variate beta, matrix-variate Dirichlet, matrix-variate gamma, matrix-variate Gauss hypergeometric, matrix-variate inverse beta, matrix-variate inverse gamma, matrix-variate Kummer beta, matrix-variate Kummer gamma, matrix-variate normal and matrix-variate two-sided power distributions.

3.1. Matrix-Variate Beta Distribution [15]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\left|\mathbf{\Omega }-\mathbf{x}\right|}^{a-\frac{p+1}{2}}{\left|\mathbf{x}\right|}^{b-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{a+b}{B}_p(a,b)}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\left|\mathbf{\Omega }-\mathbf{x}\right|}^{c-\frac{p+1}{2}}{\left|\mathbf{x}\right|}^{d-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{c+d}{B}_p(c,d)}}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $b>{\displaystyle \frac{p-1}{2}}$, $c>{\displaystyle \frac{p-1}{2}}$, $d>{\displaystyle \frac{p-1}{2}}$ and $\mathbf{\Omega }$, $\mathbf{x}$, $\mathbf{\Omega }-\mathbf{x}$ being $p\times p$ positive definite matrices. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\left|\mathbf{\Omega }\right|}^{c+d}{B}_p(c,d)}{{\left|\mathbf{\Omega }\right|}^{a+b}{B}_p(a,b)}\right]+(a-c){\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(\alpha +a,b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(b-d){\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(a,\alpha +b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}.}\hfill \end{array}$$

3.2. Matrix-Variate Dirichlet Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left({\mathbf{x}}_1,\dots ,{\mathbf{x}}_n\right)=\frac{1}{B_p\left(a_1,\dots ,a_n;a_{n+1}\right)}{\left|{\mathbf{x}}_1\right|}^{a_1-p}\cdots {\left|{\mathbf{x}}_n\right|}^{a_n-p}{\left|{\mathbf{I}}_p-\sum _{i=1}^n{\mathbf{x}}_i\right|}^{a_{n+1}-p}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left({\mathbf{x}}_1,\dots ,{\mathbf{x}}_n\right)=\frac{1}{B_p\left(b_1,\dots ,b_n;b_{n+1}\right)}{\left|{\mathbf{x}}_1\right|}^{b_1-p}\cdots {\left|{\mathbf{x}}_n\right|}^{b_n-p}{\left|{\mathbf{I}}_p-\sum _{i=1}^n{\mathbf{x}}_i\right|}^{b_{n+1}-p}}\hfill \end{array}$$

for $a_1>{\displaystyle \frac{p-1}{2}},\dots ,a_{n+1}>{\displaystyle \frac{p-1}{2}}$, $b_1>{\displaystyle \frac{p-1}{2}},\dots ,b_{n+1}>{\displaystyle \frac{p-1}{2}}$ and ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n$, ${\mathbf{I}}_p-{\mathbf{x}}_1-\cdots -{\mathbf{x}}_n$ being $p\times p$ positive definite matrices. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{B_p\left(b_1,\dots ,b_n;b_{n+1}\right)}{B_p\left(a_1,\dots ,a_n;a_{n+1}\right)}\right]+\left(a_{n+1}-b_{n+1}\right){\left.\frac{\partial }{\partial \alpha }\frac{B_p\left(a_1,\dots ,a_n;a_{n+1}+\alpha \right)}{B_p\left(a_1,\dots ,a_n;a_{n+1}\right)}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +\sum _{i=1}^n\left(a_i-b_i\right){\left.\frac{\partial }{\partial \alpha }\frac{B_p\left(a_1,\dots ,a_i+\alpha ,\dots ,a_n;a_{n+1}\right)}{B_p\left(a_1,\dots ,a_i,\dots ,a_n;a_{n+1}\right)}\right|}_{\alpha =0}.}\hfill \end{array}$$

3.3. Matrix-Variate Gamma Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\left|{\mathbf{\Sigma }}_1\right|}^{-a}}{b^{ap}{\mathsf{\Gamma }}_p\left(a\right)}{\left|\mathbf{x}\right|}^{a-\frac{p+1}{2}}exp\left[\mathrm{tr}\left(-\frac{1}{b}{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)\right]}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\left|{\mathbf{\Sigma }}_2\right|}^{-c}}{d^{cp}{\mathsf{\Gamma }}_p\left(c\right)}{\left|\mathbf{x}\right|}^{c-\frac{p+1}{2}}exp\left[\mathrm{tr}\left(-\frac{1}{d}{\mathbf{\Sigma }}_2^{-1}\mathbf{x}\right)\right]}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $b>0$, $c>{\displaystyle \frac{p-1}{2}}$, $d>0$ and $\mathbf{x}$, ${\mathbf{\Sigma }}_1$, ${\mathbf{\Sigma }}_2$ being $p\times p$ positive definite symmetric matrices. The corresponding KLD is

$$\begin{array}{c}{\displaystyle KLD=log\left[\frac{d^{pc}{\mathsf{\Gamma }}_p\left(c\right)}{b^{pa}{\mathsf{\Gamma }}_p\left(a\right)}\frac{{\left|{\mathbf{\Sigma }}_2\right|}^c}{{\left|{\mathbf{\Sigma }}_1\right|}^a}\right]+\frac{a-c}{{\mathsf{\Gamma }}_p\left(a\right)}{\left.\frac{\partial }{\partial \alpha }\left[{\left|{\mathbf{\Sigma }}_1\right|}^{\alpha }{b}^{p\alpha }{\mathsf{\Gamma }}_p(a+\alpha )\right]\right|}_{\alpha =0}+\frac{2ap}{b}-\mathrm{tr}\left[-\frac{2a}{d}{\mathbf{\Sigma }}_2^{-1}{\mathbf{\Sigma }}_1\right].}\hfill \end{array}$$

3.4. Matrix-Variate Gauss Hypergeometric Distribution [16]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\left|\mathbf{x}\right|}^{a-\frac{p+1}{2}}{\left|{\mathbf{I}}_p-\mathbf{x}\right|}^{b-\frac{p+1}{2}}}{B_p(a,b){ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right){\left|{\mathbf{I}}_p+\mathbf{B}\mathbf{x}\right|}^c}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\left|\mathbf{x}\right|}^{d-\frac{p+1}{2}}{\left|{\mathbf{I}}_p-\mathbf{x}\right|}^{e-\frac{p+1}{2}}}{B_p(d,e){ }_2{F}_1\left(d,f;d+e;-\mathbf{B}\right){\left|{\mathbf{I}}_p+\mathbf{B}\mathbf{x}\right|}^f}}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $b>{\displaystyle \frac{p-1}{2}}$, $0\le c<\infty$, $d>{\displaystyle \frac{p-1}{2}}$, $e>{\displaystyle \frac{p-1}{2}}$, $0\le f<\infty$ and $\mathbf{x}$, ${\mathbf{I}}_p-\mathbf{x}$, $\mathbf{B}$, ${\mathbf{I}}_p+\mathbf{B}$ being $p\times p$ positive definite matrices, where ${\mathsf{\Gamma }}_p\left(c\right)$ and ${\mathsf{\Gamma }}_p\left(f\right)$ are assumed to exist. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{B_p(d,e){ }_2{F}_1\left(d,f;d+e;-\mathbf{B}\right)}{B_p(a,b){ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right]}\hfill \\ \hfill & {\displaystyle +(a-d){\left.\frac{\partial }{\partial \alpha }\frac{B_p(a+\alpha ,b){ }_2{F}_1\left(a+\alpha ,c;a+b+\alpha ;-\mathbf{B}\right)}{B_p(a,b){ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(b-e){\left.\frac{\partial }{\partial \alpha }\frac{B_p(a,b+\alpha ){ }_2{F}_1\left(a,c;a+b+\alpha ;-\mathbf{B}\right)}{B_p(a,b){ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(f-c){\left.\frac{\partial }{\partial \alpha }\frac{{ }_2{F}_1\left(a,c-\alpha ;a+b;-\mathbf{B}\right)}{{ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right|}_{\alpha =0}.}\hfill \end{array}$$

3.5. Matrix-Variate Inverse Beta Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\left|\mathbf{\Omega }+\mathbf{x}\right|}^{-a-b}{\left|\mathbf{x}\right|}^{b-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{-a}{B}_p(a,b)}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\left|\mathbf{\Omega }+\mathbf{x}\right|}^{-c-d}{\left|\mathbf{x}\right|}^{d-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{-c}{B}_p(c,d)}}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $b>{\displaystyle \frac{p-1}{2}}$, $c>{\displaystyle \frac{p-1}{2}}$, $d>{\displaystyle \frac{p-1}{2}}$ and $\mathbf{x}$, $\mathbf{\Omega }$, $\mathbf{\Omega }+\mathbf{x}$ being $p\times p$ positive definite matrices. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\left|\mathbf{\Omega }\right|}^{a-c}{B}_p(c,d)}{B_p(a,b)}\right]+(c+d-a-b){\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(a-\alpha ,b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(b-d){\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(a-\alpha ,\alpha +b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}.}\hfill \end{array}$$

3.6. Matrix-Variate Inverse Gamma Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\left|{\mathbf{\Sigma }}_1\right|}^a}{b^{ap}{\mathsf{\Gamma }}_p\left(a\right)}{\left|\mathbf{x}\right|}^{-a-\frac{p+1}{2}}exp\left[\mathrm{tr}\left(-\frac{1}{b}{\mathbf{\Sigma }}_1{\mathbf{x}}^{-1}\right)\right]}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\left|{\mathbf{\Sigma }}_2\right|}^c}{d^{cp}{\mathsf{\Gamma }}_p\left(c\right)}{\left|\mathbf{x}\right|}^{-c-\frac{p+1}{2}}exp\left[\mathrm{tr}\left(-\frac{1}{d}{\mathbf{\Sigma }}_2{\mathbf{x}}^{-1}\right)\right]}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $b>0$, $c>{\displaystyle \frac{p-1}{2}}$, $d>0$ and $\mathbf{x}$, ${\mathbf{\Sigma }}_1$, ${\mathbf{\Sigma }}_2$ being $p\times p$ positive definite symmetric matrices. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{d^{pc}{\mathsf{\Gamma }}_p\left(c\right)}{b^{pa}{\mathsf{\Gamma }}_p\left(a\right)}\frac{{\left|{\mathbf{\Sigma }}_1\right|}^a}{{\left|{\mathbf{\Sigma }}_2\right|}^c}\right]+\frac{c-a}{{\mathsf{\Gamma }}_p\left(a\right)}{\left.\frac{\partial }{\partial \alpha }\left[\frac{{\left|{\mathbf{\Sigma }}_1\right|}^{\alpha }{\mathsf{\Gamma }}_p(a-\alpha )}{b^{p\alpha }}\right]\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +\frac{2a}{d}\mathrm{tr}\left[{\mathbf{\Sigma }}_2{\mathbf{\Sigma }}_1^{-1}\right]-\frac{2ap}{b}.}\hfill \end{array}$$

3.7. Matrix-Variate Kummer Beta Distribution [17]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\left|\mathbf{x}\right|}^{a-\frac{p+1}{2}}{\left|{\mathbf{I}}_p-\mathbf{x}\right|}^{b-\frac{p+1}{2}}exp\left[-\mathrm{tr}\left({\mathbf{B}}_1\mathbf{x}\right)\right]}{B_p(a,b){ }_1{F}_1\left(a,a+b;-{\mathbf{B}}_1\right)}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\left|\mathbf{x}\right|}^{c-\frac{p+1}{2}}{\left|{\mathbf{I}}_p-\mathbf{x}\right|}^{d-\frac{p+1}{2}}exp\left[-\mathrm{tr}\left({\mathbf{B}}_2\mathbf{x}\right)\right]}{B_p(c,d){ }_1{F}_1\left(c;c+d;-{\mathbf{B}}_2\right)}}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $b>{\displaystyle \frac{p-1}{2}}$, $c>{\displaystyle \frac{p-1}{2}}$, $d>{\displaystyle \frac{p-1}{2}}$ and $\mathbf{x}$, ${\mathbf{I}}_p-\mathbf{x}$, ${\mathbf{B}}_1$, ${\mathbf{B}}_2$ being $p\times p$ positive definite matrices. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{B_p(c,d){ }_1{F}_1\left(c;c+d;-{\mathbf{B}}_2\right)}{B_p(a,b){ }_1{F}_1\left(a,a+b;-{\mathbf{B}}_1\right)}\right]}\hfill \\ \hfill & {\displaystyle +(a-c){\left.\frac{\partial }{\partial \alpha }\frac{B_p(a+\alpha ,b){ }_1{F}_1\left(a+\alpha ;a+b+\alpha ;-{\mathbf{B}}_1\right)}{B_p(a,b){ }_1{F}_1\left(a;a+b;-{\mathbf{B}}_1\right)}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(b-d){\left.\frac{\partial }{\partial \alpha }\frac{B_p(a,b+\alpha ){ }_1{F}_1\left(a;a+b+\alpha ;-{\mathbf{B}}_1\right)}{B_p(a,b){ }_1{F}_1\left(a;a+b;-{\mathbf{B}}_1\right)}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +\mathrm{tr}\left[\left({\mathbf{B}}_2-{\mathbf{B}}_1\right){\left.\frac{\partial }{\partial \mathbf{z}}\frac{{ }_1{F}_1\left(a;a+b;\mathbf{z}-{\mathbf{B}}_1\right)}{{ }_1{F}_1\left(a;a+b;-{\mathbf{B}}_1\right)}\right|}_{\mathbf{z}=\mathbf{0}}\right].}\hfill \end{array}$$

3.8. Matrix-Variate Kummer Gamma Distribution [18]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{{\left|\mathbf{x}\right|}^{a-\frac{p+1}{2}}{\left|{\mathbf{I}}_p+\mathbf{x}\right|}^{-b}exp\left[-\mathrm{tr}\left({\mathbf{B}}_1\mathbf{x}\right)\right]}{{\mathsf{\Gamma }}_p\left(a\right){\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{{\left|\mathbf{x}\right|}^{c-\frac{p+1}{2}}{\left|{\mathbf{I}}_p+\mathbf{x}\right|}^{-d}exp\left[-\mathrm{tr}\left({\mathbf{B}}_2\mathbf{x}\right)\right]}{{\mathsf{\Gamma }}_p\left(c\right){\mathsf{\Psi }}_1\left(c;c-d+\frac{p+1}{2};{\mathbf{B}}_2\right)}}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $-\infty <b<\infty$, $c>{\displaystyle \frac{p-1}{2}}$, $-\infty <d<\infty$ and $\mathbf{x}$, ${\mathbf{I}}_p+\mathbf{x}$, ${\mathbf{B}}_1$, ${\mathbf{B}}_2$ being $p\times p$ positive definite matrices, where ${\mathsf{\Psi }}_1\left(a;a-b+{\displaystyle \frac{p+1}{2}};{\mathbf{B}}_1\right)$ and ${\mathsf{\Psi }}_1\left(c;c-d+{\displaystyle \frac{p+1}{2}};{\mathbf{B}}_2\right)$ denote Kummer functions with matrix arguments and the parameters are chosen such that these functions exist. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\mathsf{\Gamma }}_p\left(c\right){\mathsf{\Psi }}_1\left(c;c-d+\frac{p+1}{2};{\mathbf{B}}_2\right)}{{\mathsf{\Gamma }}_p\left(a\right){\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right]}\hfill \\ \hfill & {\displaystyle +(a-c){\left.\frac{\partial }{\partial \alpha }\frac{{\mathsf{\Gamma }}_p(a+\alpha ){\mathsf{\Psi }}_1\left(a+\alpha ;a-b+\alpha +\frac{p+1}{2};{\mathbf{B}}_1\right)}{{\mathsf{\Gamma }}_p\left(a\right){\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(d-p){\left.\frac{\partial }{\partial \alpha }\frac{{\mathsf{\Psi }}_1\left(a;a-b+\alpha +\frac{p+1}{2};{\mathbf{B}}_1\right)}{{\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +\mathrm{tr}\left[\left({\mathbf{B}}_2-{\mathbf{B}}_1\right){\left.\frac{\partial }{\partial \mathbf{z}}\frac{{\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1-\mathbf{z}\right)}{{\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right|}_{\mathbf{z}=\mathbf{0}}\right].}\hfill \end{array}$$

3.9. Matrix-Variate Normal Distribution

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=\frac{1}{{\left(2\pi \right)}^{\frac{np}{2}}{\left|{\mathbf{V}}_1\right|}^{\frac{n}{2}}{\left|{\mathbf{U}}_1\right|}^{\frac{p}{2}}}exp\left\{-\frac{1}{2}\mathrm{tr}\left[{\mathbf{V}}_1^{-1}{\left(\mathbf{x}-{\mathbf{M}}_1\right)}^T{\mathbf{U}}_1^{-1}\left(\mathbf{x}-{\mathbf{M}}_1\right)\right]\right\}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=\frac{1}{{\left(2\pi \right)}^{\frac{np}{2}}{\left|{\mathbf{V}}_2\right|}^{\frac{n}{2}}{\left|{\mathbf{U}}_2\right|}^{\frac{p}{2}}}exp\left\{-\frac{1}{2}\mathrm{tr}\left[{\mathbf{V}}_2^{-1}{\left(\mathbf{x}-{\mathbf{M}}_2\right)}^T{\mathbf{U}}_2^{-1}\left(\mathbf{x}-{\mathbf{M}}_2\right)\right]\right\}}\hfill \end{array}$$

for ${\mathbf{U}}_1$, ${\mathbf{U}}_2$ being positive definite symmetric matrices of dimension $n\times n$, ${\mathbf{V}}_1$, ${\mathbf{V}}_2$ being positive definite symmetric matrices of dimension $p\times p$ and ${\mathbf{M}}_1$, ${\mathbf{M}}_2$ being matrices of dimension $n\times p$. The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\left|{\mathbf{V}}_2\right|}^{\frac{n}{2}}{\left|{\mathbf{U}}_2\right|}^{\frac{p}{2}}}{{\left|{\mathbf{V}}_1\right|}^{\frac{n}{2}}{\left|{\mathbf{U}}_1\right|}^{\frac{p}{2}}}\right]+\frac{1}{2}\mathrm{tr}\left({\mathbf{U}}_1\right)\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{V}}_1{\mathbf{U}}_2^{-1}\right)+\frac{1}{2}\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_1^T{\mathbf{M}}_1{\mathbf{U}}_2^{-1}\right)}\hfill \\ \hfill & {\displaystyle -\frac{1}{2}\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_1^T{\mathbf{M}}_2{\mathbf{U}}_2^{-1}\right)-\frac{1}{2}\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_2^T{\mathbf{M}}_1{\mathbf{U}}_2^{-1}\right)}\hfill \\ \hfill & {\displaystyle +\frac{1}{2}\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_2^T{\mathbf{M}}_2{\mathbf{U}}_2^{-1}\right)-\frac{1}{2}\mathrm{tr}\left({\mathbf{U}}_1\right)\mathrm{tr}\left({\mathbf{U}}_1^{-1}\right).}\hfill \end{array}$$

3.10. Matrix-Variate Two-Sided Power Distribution [19]

Consider the joint probability density functions

$$\begin{array}{c}{\displaystyle f_1\left(\mathbf{x}\right)=C\left(a\right)\left\{\begin{array}{cc}{\displaystyle {\left|\mathbf{x}\right|}^{a-\frac{p+1}{2}}{\left|\mathbf{B}\right|}^{-a+\frac{p+1}{2}},}\hfill & {\mathbf{0}}_p<\mathbf{x}\le \mathbf{B},\hfill \\ \\ {\displaystyle {\left|{\mathbf{I}}_p-\mathbf{x}\right|}^{a-\frac{p+1}{2}}{\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{-a+\frac{p+1}{2}},}\hfill & \mathbf{B}\le \mathbf{x}\le {\mathbf{I}}_p\hfill \end{array}\right.}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle f_2\left(\mathbf{x}\right)=C\left(b\right)\left\{\begin{array}{cc}{\displaystyle {\left|\mathbf{x}\right|}^{b-\frac{p+1}{2}}{\left|\mathbf{B}\right|}^{-b+\frac{p+1}{2}},}\hfill & {\mathbf{0}}_p<\mathbf{x}\le \mathbf{B},\hfill \\ \\ {\displaystyle {\left|{\mathbf{I}}_p-\mathbf{x}\right|}^{b-\frac{p+1}{2}}{\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{-b+\frac{p+1}{2}},}\hfill & \mathbf{B}\le \mathbf{x}\le {\mathbf{I}}_p,\hfill \end{array}\right.}\hfill \end{array}$$

for $a>{\displaystyle \frac{p-1}{2}}$, $b>{\displaystyle \frac{p-1}{2}}$ and $\mathbf{x}$, ${\mathbf{I}}_p-\mathbf{x}$, $\mathbf{B}$, ${\mathbf{I}}_p-\mathbf{B}$ being $p\times p$ positive definite matrices, where

$$\begin{array}{c}{\displaystyle C\left(a\right)={\left|\mathbf{B}\right|}^{\frac{p+1}{2}}{B}_p\left(a,\frac{p+1}{2}\right)+{\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{\frac{p+1}{2}}{B}_p\left(\frac{p+1}{2},a\right)}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle C\left(b\right)={\left|\mathbf{B}\right|}^{\frac{p+1}{2}}{B}_p\left(b,\frac{p+1}{2}\right)+{\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{\frac{p+1}{2}}{B}_p\left(\frac{p+1}{2},b\right).}\hfill \end{array}$$

The corresponding KLD is

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{C\left(a\right)}{C\left(b\right)}\right]+(a-b){\left.\frac{\partial }{\partial \alpha }\left\{{\left|\mathbf{B}\right|}^{\alpha +\frac{p+1}{2}}{B}_p\left(a+\alpha ,\frac{p+1}{2}\right)\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(a-b){\left.\frac{\partial }{\partial \alpha }\left\{{\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{\alpha +\frac{p+1}{2}}{B}_p\left(\frac{p+1}{2},a+\alpha \right)\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle +(b-a){\left|\mathbf{B}\right|}^{\frac{p+1}{2}}log\left|\mathbf{B}\right|B_p\left(a,\frac{p+1}{2}\right)}\hfill \\ \hfill & {\displaystyle +(b-a){\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{\frac{p+1}{2}}log\left|{\mathbf{I}}_p-\mathbf{B}\right|B_p\left(\frac{p+1}{2},a\right).}\hfill \end{array}$$

4. Proofs

Before presenting the proofs of the expressions in Section 2 and Section 3, we state a lemma and give its proof.

4.1. A Technical Lemma

Lemma 1. 

Let

$$\begin{array}{c}{\displaystyle I\left(a_1,\dots ,a_p,t_1,\dots ,t_p,b\right)={\int }_{{\mathbb{R}}^p}\frac{{\displaystyle exp\left(-\sum _{j=1}^p{t}_j{x}_j\right)}}{{\displaystyle {\left[1+\sum _{j=1}^{p}exp\left(-a_j{x}_j\right)\right]}^b}}d{x}_1\cdots d{x}_p}\hfill \end{array}$$

for $t_j>0$, $a_j>0$, $j=1,2,\dots ,p$ and $b>0$. Then,

$$\begin{array}{c}{\displaystyle I\left(a_1,\dots ,a_p,t_1,\dots ,t_p,b\right)=\frac{1}{a_1\cdots a_p\mathsf{\Gamma }\left(b\right)}\left[\prod _{j=1}^p\mathsf{\Gamma }\left(\frac{t_j}{a_j}\right)\right]\mathsf{\Gamma }\left(b-\sum _{j=1}^p\frac{t_j}{a_j}\right)}\hfill \end{array}$$

provided that $b-{\sum }_{j=1}^p{\displaystyle \frac{t_j}{a_j}}>0$.

Proof. 

Setting $y_j=exp\left(-a_j{x}_j\right)$ and assuming the conditions in the lemma, we can write

$$\begin{array}{cc}\hfill & {\displaystyle I\left(a_1,\dots ,a_p,t_1,\dots ,t_p,b\right)}\hfill \\ \hfill & {\displaystyle =\frac{1}{\mathsf{\Gamma }\left(b\right)}{\int }_{{\mathbb{R}}^p}{\int }_0^{\infty }{t}^{b-1}exp\left\{-\left[1+\sum _{j=1}^{p}exp\left(-a_j{x}_j\right)\right]t\right\}exp\left(-\sum _{j=1}^p{t}_j{x}_j\right)dtd{x}_1\cdots d{x}_p}\hfill \\ \hfill & {\displaystyle =\frac{1}{\mathsf{\Gamma }\left(b\right)}{\int }_0^{\infty }{t}^{b-1}{e}^{-t}\prod _{j=1}^p\left\{{\int }_{-\infty }^{\infty }exp\left[-t_j{x}_j-texp\left(-a_j{x}_j\right)\right]d{x}_j\right\}dt}\hfill \\ \hfill & {\displaystyle =\frac{1}{a_1\cdots a_p\mathsf{\Gamma }\left(b\right)}{\int }_0^{\infty }{t}^{b-1}{e}^{-t}\prod _{j=1}^p\left[{\int }_0^{\infty }{y}_j^{\frac{t_j}{a_j}-1}exp\left(-t{y}_j\right)d{y}_j\right]dt}\hfill \\ \hfill & {\displaystyle =\frac{1}{a_1\cdots a_p\mathsf{\Gamma }\left(b\right)}\left[\prod _{j=1}^p\mathsf{\Gamma }\left(\frac{t_j}{a_j}\right)\right]{\int }_0^{\infty }{t}^{b-{\sum }_{j=1}^p\frac{t_j}{a_j}-1}{e}^{-t}dt.}\hfill \end{array}$$

The result follows. □

4.2. Proof for Section 2.1

The corresponding KLD can be expressed as

$$\begin{array}{c}{\displaystyle KLD=log\left[\frac{B\left(b_1,\dots ,b_{K-1};b_K\right)}{B\left(a_1,\dots ,a_{K-1};a_K\right)}\right]+\sum _{i=1}^K\left(a_i-b_i\right)E\left[log{X}_i\right].}\hfill \end{array}$$

(2)

It is easy to show that

$$\begin{array}{c}{\displaystyle E\left[log{X}_i\right]=\psi \left(a_i\right)-\psi \left(a_1+\cdots +a_K\right),}\hfill \end{array}$$

so (2) reduces to the required.

4.3. Proof for Section 2.2

The corresponding KLD can be expressed as

$$\begin{array}{c}{\displaystyle KLD=log\left[\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{\beta }\right)}{\beta \mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}\right]+\frac{1}{2}log\frac{\left|{\mathbf{V}}_2\right|}{\left|{\mathbf{V}}_1\right|}+E\left[{\left({\mathbf{X}}^T{\mathbf{V}}_2^{-1}\mathbf{X}\right)}^{\frac{\beta }{2}}\right]-E\left[{\left({\mathbf{X}}^T{\mathbf{V}}_1^{-1}\mathbf{X}\right)}^{\frac{\alpha }{2}}\right].}\hfill \end{array}$$

(3)

The second expectation in (3) can be expressed as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{V}}_1^{-1}\mathbf{X}\right)}^{\frac{\alpha }{2}}\right]}\hfill & {\displaystyle =\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\left|{\mathbf{V}}_1\right|}^{-\frac{1}{2}}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{x}}^T{\mathbf{V}}_1^{-1}\mathbf{x}\right)}^{\frac{\alpha }{2}}exp\left\{-{\left({\mathbf{x}}^T{\mathbf{V}}_1^{-1}\mathbf{x}\right)}^{\frac{\alpha }{2}}\right\}d\mathbf{x}}\hfill \\ \hfill & {\displaystyle =\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{y}}^T\mathbf{y}\right)}^{\frac{\alpha }{2}}exp\left\{-{\left({\mathbf{y}}^T\mathbf{y}\right)}^{\frac{\alpha }{2}}\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{\alpha }{2\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_0^{\infty }{u}^{\frac{p}{2}+\frac{\alpha }{2}-1}exp\left(-u^{\frac{\alpha }{2}}\right)du}\hfill \\ \hfill & {\displaystyle =\frac{1}{\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_0^{\infty }{t}^{\frac{p}{\alpha }}exp(-t)dt}\hfill \\ \hfill & {\displaystyle =\frac{p}{\alpha },}\hfill \end{array}$$

where $\mathbf{y}={\mathbf{V}}_1^{-{\displaystyle \frac{1}{2}}}\mathbf{x}$, $u={\mathbf{y}}^T\mathbf{y}$ and $t=u^{{\displaystyle \frac{\alpha }{2}}}$.

Let $\mathbf{V}={\mathbf{V}}_1^{{\displaystyle \frac{1}{2}}}{\mathbf{V}}_2^{-1}{\mathbf{V}}_1^{{\displaystyle \frac{1}{2}}}$ and $\mathbf{V}=\mathbf{P}\mathbf{D}{\mathbf{P}}^{-1}$, where $\mathbf{P}$ is an orthonormal matrix composed of eigenvectors of $\mathbf{V}$ and $\mathbf{D}$ is a diagonal matrix composed of eigenvalues say ${\lambda }_i$ of $\mathbf{V}$. Then, the first expectation in (3) can be expressed as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{V}}_2^{-1}\mathbf{X}\right)}^{\frac{\beta }{2}}\right]}\hfill & {\displaystyle =\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_{{\mathbb{R}}^p}{\left[\mathrm{tr}\left({\mathbf{DP}}^T{\mathbf{yy}}^T\mathbf{P}\right)\right]}^{\frac{\beta }{2}}exp\left\{-{\left({\mathbf{y}}^T\mathbf{y}\right)}^{\frac{\alpha }{2}}\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{x}}^T\mathbf{D}\mathbf{x}\right)}^{\frac{\beta }{2}}exp\left\{-{\left({\mathbf{x}}^T\mathbf{x}\right)}^{\frac{\alpha }{2}}\right\}d\mathbf{x}}\hfill \\ \hfill & {\displaystyle =\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_{\mathbb{R}}\cdots {\int }_{\mathbb{R}}{\left(\sum _i^p{\lambda }_i{x}_i^2\right)}^{\frac{\beta }{2}}exp\left\{-{\left(\sum _i^p{x}_i^2\right)}^{\frac{\alpha }{2}}\right\}d\mathbf{x},}\hfill \end{array}$$

(4)

where $\mathbf{y}={\mathbf{V}}_1^{-{\displaystyle \frac{1}{2}}}\mathbf{x}$ and $\mathbf{z}={\mathbf{P}}^T\mathbf{y}$. Using the pseudo-polar transformation $z_1=rsin{\theta }_1$, $z_2=rcos{\theta }_1$$sin{\theta }_2$, …, $z_p=rcos{\theta }_1$$cos{\theta }_2$⋯$cos{\theta }_{p-1}$ https://en.wikipedia.org/wiki/Polar_coordinate_system (accessed on 1 July 2024), (4) can be expressed as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{V}}_2^{-1}\mathbf{X}\right)}^{\frac{\beta }{2}}\right]}\hfill & {\displaystyle =\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{2{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_0^{\infty }{r}^{p-1}{\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}\cdots {\int }_{-\pi }^{\pi }{\left[r^2\left({\lambda }_1{sin}^2{\theta }_1+\cdots +{\lambda }_p{cos}^2{\theta }_1\cdots {cos}^2{\theta }_{p-1}\right)\right]}^{\frac{\beta }{2}}}\hfill \\ \hfill & {\displaystyle \times exp\left[-{\left(r^2\right)}^{\frac{\alpha }{2}}\right]\left[\prod _{j=1}^{p-1}{\left|cos{\theta }_j\right|}^{p-j-1}\right]dr\prod _{j=1}^{p-1}d{\theta }_j}\hfill \\ \hfill & {\displaystyle =\frac{\alpha \mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_0^{\infty }{r}^{p+\beta -1}exp\left(-r^{\alpha }\right){\int }_0^1\cdots {\int }_0^1\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]}\hfill \\ \hfill & {\displaystyle \times {\left[{\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)\right]}^{\frac{\beta }{2}}d{x}_1\cdots d{x}_{p-1}dr}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{p+\beta }{2}\right)}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}{\left[{\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)\right]}^{\frac{\beta }{2}}d{x}_1\cdots d{x}_{p-1},}\hfill \end{array}$$

(5)

where $x_i={cos}^2{\theta }_i$ and ${\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)={\lambda }_1+\left({\lambda }_2-{\lambda }_1\right)x_1+\cdots +\left({\lambda }_p-{\lambda }_{p-1}\right)x_1{x}_2\cdots x_{p-1}$. Provided that $\left|{\lambda }_1\right|\ge \left|\left({\lambda }_2-{\lambda }_1\right)x_1+\cdots +\left({\lambda }_p-{\lambda }_{p-1}\right)x_1{x}_2\cdots x_{p-1}\right|$ holds, we can apply the generalized multinomial theorem to calculate (5) as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{V}}_2^{-1}\mathbf{X}\right)}^{\frac{\beta }{2}}\right]}\hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{p+\beta }{2}\right)}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times \sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left(\genfrac{}{}{0pt}{}{\frac{\beta }{2}}{r_1}\right)\left(\genfrac{}{}{0pt}{}{r_1}{r_2}\right)\cdots \left(\genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}\right)}\hfill \\ \hfill & {\displaystyle \times {\lambda }_1^{\frac{\beta }{2}-r_1}{\left[\left({\lambda }_2-{\lambda }_1\right)x_1\right]}^{r_1-r_2}{\left[\left({\lambda }_3-{\lambda }_2\right)x_1{x}_2\right]}^{r_2-r_3}}\hfill \\ \hfill & {\displaystyle \cdots {\left[\left({\lambda }_{p-1}-{\lambda }_{p-2}\right)x_1{x}_2\cdots x_{p-2}\right]}^{r_{p-2}-r_{p-1}}{\left[\left({\lambda }_p-{\lambda }_{p-1}\right)x_1{x}_2\cdots x_{p-1}\right]}^{r_{p-1}}d{x}_1\cdots d{x}_{p-1}}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{p+\beta }{2}\right)}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times \sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left(\genfrac{}{}{0pt}{}{\frac{\beta }{2}}{r_1}\right)\left(\genfrac{}{}{0pt}{}{r_1}{r_2}\right)\cdots \left(\genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}\right){\lambda }_1^{\frac{\beta }{2}-r_1}}\hfill \\ \hfill & {\displaystyle \times \prod _{j=1}^{p-1}{\left[\left({\lambda }_{j+1}-{\lambda }_j\right)\left(\prod _{\ell =1}^j{x}_{\ell }\right)\right]}^{r_j-r_{j+1}}d{x}_1\cdots d{x}_{p-1}}\hfill \\ {\displaystyle }\hfill & ={\displaystyle \frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{p+\beta }{2}\right)}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{p}{\alpha }\right)}}\sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left({\displaystyle \genfrac{}{}{0pt}{}{\frac{\beta }{2}}{r_1}}\right)\left({\displaystyle \genfrac{}{}{0pt}{}{r_1}{r_2}}\right)\cdots \left({\displaystyle \genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}}\right){\lambda }_1^{{\displaystyle \frac{\beta }{2}}-r_1}\hfill \\ \hfill & {\displaystyle \times \prod _{j=1}^{p-1}\left[{\left({\lambda }_{j+1}-{\lambda }_j\right)}^{r_j-r_{j+1}}B\left(r_j+\frac{p-j}{2},\frac{1}{2}\right)\right]}\hfill \end{array}$$

provided that the infinite sum converges. Hence, the required.

4.4. Proof for Section 2.3

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\displaystyle \mathsf{\Gamma }\left(a_1+\cdots +a_{K+1}\right)\mathsf{\Gamma }\left(b_1\right)\cdots \mathsf{\Gamma }\left(b_{K+1}\right)}}{{\displaystyle \mathsf{\Gamma }\left(b_1+\cdots +b_{K+1}\right)\mathsf{\Gamma }\left(a_1\right)\cdots \mathsf{\Gamma }\left(a_{K+1}\right)}}\right]+\sum _{i=1}^K\left(a_i-b_i\right)E\left[log{X}_i\right]}\hfill \\ \hfill & {\displaystyle +\left(b_1+\cdots +b_{K+1}-a_1-\cdots -a_{K+1}\right)E\left[log\left(1+\sum _{i=1}^K{X}_i\right)\right].}\hfill \end{array}$$

(6)

It is easy to show that

$$\begin{array}{c}{\displaystyle E\left[log{X}_i\right]=\psi \left(a_i\right)-\psi \left(a_{K+1}\right)}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left(1+\sum _{i=1}^K{X}_i\right)\right]=\psi \left(a_1+\cdots +a_{K+1}\right)-\psi \left(a_{K+1}\right),}\hfill \end{array}$$

so (6) reduces to the required.

4.5. Proof for Section 2.4

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{C\left(a_1,\dots ,a_K,b,c\right)}{C\left(d_1,\dots ,d_K,e,f\right)}\right]+\sum _{i=1}^K\left(a_i-d_i\right)E\left[log{X}_i\right]+(b-e)E\left[log\left(1-\sum _{i=1}^K{X}_i\right)\right]}\hfill \\ \hfill & {\displaystyle -(c-f)E\left[log\left(1+\sum _{i=1}^K{X}_i\right)\right].}\hfill \end{array}$$

(7)

It is easy to show that

$$\begin{array}{c}{\displaystyle E\left[log{X}_i\right]={\left.\frac{\partial }{\partial \alpha }\left[\frac{C\left(a_1,\dots ,a_i,\dots ,a_K,b,c\right)}{C\left(a_1,\dots ,a_i+\alpha ,\dots ,a_K,b,c\right)}\right]\right|}_{\alpha =0},}\hfill \end{array}$$

$$\begin{array}{c}{\displaystyle E\left[log\left(1-\sum _{i=1}^K{X}_i\right)\right]={\left.\frac{\partial }{\partial \alpha }\left[\frac{C\left(a_1,\dots ,a_i,\dots ,a_K,b,c\right)}{C\left(a_1,\dots ,a_K,b+\alpha ,c\right)}\right]\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left(1+\sum _{i=1}^K{X}_i\right)\right]={\left.\frac{\partial }{\partial \alpha }\left[\frac{C\left(a_1,\dots ,a_i,\dots ,a_K,b,c\right)}{C\left(a_1,\dots ,a_K,b,c-\alpha \right)}\right]\right|}_{\alpha =0},}\hfill \end{array}$$

so (7) reduces to the required.

4.6. Proof for Section 2.5

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{a{q}^{\frac{2N+p-2}{2a}}\mathsf{\Gamma }\left(\frac{2M+p-2}{2b}\right)}{b{s}^{\frac{2M+p-2}{2b}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\right]+\frac{1}{2}log\frac{\left|{\mathbf{\Sigma }}_2\right|}{\left|{\mathbf{\Sigma }}_1\right|}+(N-1)E\left[log\left({\mathbf{X}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{X}\right)\right]}\hfill \\ \hfill & {\displaystyle -qE\left[{\left({\mathbf{X}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{X}\right)}^a\right]-(M-1)E\left[log\left({\mathbf{X}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{X}\right)\right]+sE\left[{\left({\mathbf{X}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{X}\right)}^b\right].}\hfill \end{array}$$

(8)

The second expectation in (8) can be calculated as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{X}\right)}^a\right]}\hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\left|{\mathbf{\Sigma }}_1\right|}^{-\frac{1}{2}}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^{a+N-1}exp\left\{-q{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^a\right\}d\mathbf{x}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{y}}^T\mathbf{y}\right)}^{a+N-1}exp\left\{-q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{1}{q\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }{t}^{\frac{2N+p-2}{2a}}exp(-t)dt}\hfill \\ \hfill & {\displaystyle =\frac{2N+p-2}{2aq},}\hfill \end{array}$$

where $\mathbf{y}={\mathbf{\Sigma }}_1^{-{\displaystyle \frac{1}{2}}}\mathbf{x}$ and $t=q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a$.

Let $\mathbf{\Sigma }={\mathbf{\Sigma }}_1^{{\displaystyle \frac{1}{2}}}{\mathbf{\Sigma }}_2^{-1}{\mathbf{\Sigma }}_1^{{\displaystyle \frac{1}{2}}}$. The first expectation in (8) can be calculated as

$$\begin{array}{cc}{\displaystyle E\left[log\left({\mathbf{X}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{X}\right)\right]}\hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\left|{\mathbf{\Sigma }}_1\right|}^{-\frac{1}{2}}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^{N-1}log\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)exp\left\{-q{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^a\right\}d\mathbf{x}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{y}}^T\mathbf{y}\right)}^{N-1}log\left({\mathbf{y}}^T\mathbf{y}\right)exp\left\{-q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\frac{\partial }{\partial N}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{y}}^T\mathbf{y}\right)}^{N-1}exp\left\{-q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{q^{\frac{2N+p-2}{2a}}}{\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\frac{\partial }{\partial N}{\int }_0^{\infty }\frac{t^{\frac{2N+p-2}{2a}-1}}{q^{\frac{2N+p-2}{2a}}}exp(-t)dt}\hfill \\ \hfill & {\displaystyle =\frac{q^{\frac{2N+p-2}{2a}}}{\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\frac{\partial }{\partial N}\left[\frac{\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{q^{\frac{2N+p-2}{2a}}}\right]}\hfill \\ \hfill & {\displaystyle =\frac{1}{a}\left[\psi \left(\frac{2N+p-2}{2a}\right)-logq\right],}\hfill \end{array}$$

where $\mathbf{y}={\mathbf{\Sigma }}_1^{-{\displaystyle \frac{1}{2}}}\mathbf{x}$ and $t=q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a$.

As in Section 4.3, write $\mathbf{\Sigma }=\mathbf{P}\mathbf{D}{\mathbf{P}}^{-1}$, where $\mathbf{P}$ is an orthonormal matrix composed of eigenvectors of $\mathbf{\Sigma }$ and $\mathbf{D}$ is a diagonal matrix composed of eigenvalues say ${\lambda }_i$ of $\mathbf{\Sigma }$. Then, the fourth expectation in (8) can be expressed as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{X}\right)}^b\right]}\hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\left|{\mathbf{\Sigma }}_1\right|}^{-\frac{1}{2}}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{x}\right)}^b{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^{N-1}exp\left\{-q{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^a\right\}d\mathbf{x}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{y}}^T\mathbf{\Sigma }\mathbf{y}\right)}^b{\left({\mathbf{y}}^T\mathbf{y}\right)}^{N-1}exp\left\{-q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left[\mathrm{tr}\left(\mathbf{D}{\mathbf{P}}^T\mathbf{y}{\mathbf{y}}^T\mathbf{P}\right)\right]}^b{\left({\mathbf{y}}^T\mathbf{y}\right)}^{N-1}exp\left\{-q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{v}}^T\mathbf{D}\mathbf{v}\right)}^b{\left({\mathbf{v}}^T\mathbf{v}\right)}^{N-1}exp\left\{-q{\left({\mathbf{v}}^T\mathbf{v}\right)}^a\right\}d\mathbf{v}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left(\sum _{i=1}^p{\lambda }_i{v}_i^2\right)}^b{\left(\sum _{i=1}^p{v}_i^2\right)}^{N-1}exp\left\{-q{\left(\sum _{i=1}^p{v}_i^2\right)}^a\right\}d\mathbf{v},}\hfill \end{array}$$

(9)

where $\mathbf{y}={\mathbf{\Sigma }}_1^{-{\displaystyle \frac{1}{2}}}\mathbf{x}$ and $\mathbf{v}={\mathbf{P}}^T\mathbf{y}$.

Using the pseudo-polar transformation $v_1=rsin{\theta }_1$, $v_2=rcos{\theta }_{1}sin{\theta }_2$, …, $v_p=r$$cos{\theta }_1$$cos{\theta }_2$⋯$cos{\theta }_{p-1}$, (9) can be expressed as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{X}\right)}^b\right]}\hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }{r}^{p-1}{\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}\cdots {\int }_{-\pi }^{\pi }{\left[r^2\left({\lambda }_1{sin}^2{\theta }_1+\cdots +{\lambda }_p{cos}^2{\theta }_1\cdots {cos}^2{\theta }_{p-1}\right)\right]}^b}\hfill \\ \hfill & {\displaystyle \times {\left(r^2\right)}^{N-1}exp\left[-q{\left(r^2\right)}^a\right]\left[\prod _{j=1}^{p-1}{\left|cos{\theta }_j\right|}^{p-j-1}\right]dr\prod _{j=1}^{p-1}d{\theta }_j}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }\frac{t^{\frac{2N+p+2b-2}{2a}-1}}{q^{\frac{2N+p+2b-2}{2a}}}exp(-t){\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times {\left[{\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)\right]}^{b}d{x}_1\cdots d{x}_{p-1}dt}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{2N+p+2b-2}{2a}\right)}{{\pi }^{\frac{p}{2}}{q}^{\frac{b}{a}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times {\left[{\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)\right]}^{b}d{x}_1\cdots d{x}_{p-1},}\hfill \end{array}$$

(10)

where $x_i={cos}^2{\theta }_i$, $t=q{r}^{2a}$ and ${\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)={\lambda }_1+\left({\lambda }_2-{\lambda }_1\right)x_1+\cdots +\left({\lambda }_p-{\lambda }_{p-1}\right)$$x_1{x}_2\cdots x_{p-1}$.

Provided that $\left|{\lambda }_1\right|\ge \left|\left({\lambda }_2-{\lambda }_1\right)x_1+\cdots +\left({\lambda }_p-{\lambda }_{p-1}\right)x_1{x}_2\cdots x_{p-1}\right|$ holds, we can apply the generalized multinomial theorem to calculate (10) as

$$\begin{array}{cc}{\displaystyle E\left[{\left({\mathbf{X}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{X}\right)}^b\right]}\hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{2N+p+2b-2}{2a}\right)}{{\pi }^{\frac{p}{2}}{q}^{\frac{b}{a}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times \sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left(\genfrac{}{}{0pt}{}{b}{r_1}\right)\left(\genfrac{}{}{0pt}{}{r_1}{r_2}\right)\cdots \left(\genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}\right){\lambda }_1^{b-r_1}{\left[\left({\lambda }_2-{\lambda }_1\right)x_1\right]}^{r_1-r_2}{\left[\left({\lambda }_3-{\lambda }_2\right)x_1{x}_2\right]}^{r_2-r_3}}\hfill \\ \hfill & {\displaystyle \cdots {\left[\left({\lambda }_{p-1}-{\lambda }_{p-2}\right)x_1{x}_2\cdots x_{p-2}\right]}^{r_{p-2}-r_{p-1}}{\left[\left({\lambda }_p-{\lambda }_{p-1}\right)x_1{x}_2\cdots x_{p-1}\right]}^{r_{p-1}}d{x}_1\cdots d{x}_{p-1}}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{2N+p+2b-2}{2a}\right)}{{\pi }^{\frac{p}{2}}{q}^{\frac{b}{a}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times \sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left(\genfrac{}{}{0pt}{}{b}{r_1}\right)\left(\genfrac{}{}{0pt}{}{r_1}{r_2}\right)\cdots \left(\genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}\right)}\hfill \\ \hfill & {\displaystyle \times {\lambda }_1^{b-r_1}\left\{\prod _{j=1}^{p-1}{\left[\left({\lambda }_{j+1}-{\lambda }_j\right)\prod _{\ell =1}^j{x}_{\ell }\right]}^{r_j-r_{j+1}}\right\}d{x}_1\cdots d{x}_{p-1}}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)\mathsf{\Gamma }\left(\frac{2N+p+2b-2}{2a}\right)}{{\pi }^{\frac{p}{2}}{q}^{\frac{b}{a}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\sum _{r_1=0}^{\infty }\sum _{r_2=0}^{r_1}\cdots \sum _{r_{p-1}=0}^{r_{p-2}}\left(\genfrac{}{}{0pt}{}{b}{r_1}\right)\left(\genfrac{}{}{0pt}{}{r_1}{r_2}\right)\cdots \left(\genfrac{}{}{0pt}{}{r_{p-2}}{r_{p-1}}\right){\lambda }_1^{b-r_1}}\hfill \\ \hfill & {\displaystyle \times \prod _{j=1}^{p-1}\left[{\left({\lambda }_{j+1}-{\lambda }_j\right)}^{r_j-r_{j+1}}B\left(r_j+\frac{p-j}{2},\frac{1}{2}\right)\right]}\hfill \end{array}$$

provided that the infinite sum converges.

The third expectation in (8) can be expressed as

$$\begin{array}{cc}{\displaystyle E\left[log\left({\mathbf{X}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{X}\right)\right]}\hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\left|{\mathbf{\Sigma }}_1\right|}^{-\frac{1}{2}}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^{N-1}log\left({\mathbf{x}}^T{\mathbf{\Sigma }}_2^{-1}\mathbf{x}\right)exp\left\{-q{\left({\mathbf{x}}^T{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)}^a\right\}d\mathbf{x}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left({\mathbf{y}}^T\mathbf{y}\right)}^{N-1}log\left[\mathrm{tr}\left(\mathbf{D}{\mathbf{P}}^T\mathbf{y}{\mathbf{y}}^T\mathbf{P}\right)\right]exp\left\{-q{\left({\mathbf{y}}^T\mathbf{y}\right)}^a\right\}d\mathbf{y}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_{{\mathbb{R}}^p}{\left(\sum _{i=1}^p{v}_i^2\right)}^{N-1}log\left(\sum _{i=1}^p{\lambda }_i{v}_i^2\right)exp\left\{-q{\left(\sum _{i=1}^p{v}_i^2\right)}^a\right\}d\mathbf{v}}\hfill \\ \hfill & {\displaystyle =\frac{a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }{r}^{p-1}{\int }_{-\frac{\pi }{2}}^{\frac{\pi }{2}}\cdots {\int }_{-\pi }^{\pi }{\left(r^2\right)}^{N-1}}\hfill \\ \hfill & {\displaystyle \times log\left[r^2\left({\lambda }_1{sin}^2{\theta }_1+\cdots +{\lambda }_p{cos}^2{\theta }_1\cdots {cos}^2{\theta }_{p-1}\right)\right]}\hfill \\ \hfill & {\displaystyle \times exp\left\{-q{\left(r^2\right)}^a\right\}\left[\prod _{j=1}^{p-1}{\left|cos{\theta }_j\right|}^{p-j-1}\right]dr\prod _{j=1}^{p-1}d{\theta }_j}\hfill \\ \hfill & {\displaystyle =I_1+I_2}\hfill \end{array}$$

say, where $\mathbf{y}={\mathbf{\Sigma }}_1^{-{\displaystyle \frac{1}{2}}}\mathbf{x}$, $\mathbf{v}={\mathbf{P}}^T\mathbf{y}$,

$$\begin{array}{c}{\displaystyle I_1=\frac{2a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }{r}^{p+2N-3}log\left(r^2\right)exp\left(-q{r}^{2a}\right)}\hfill \\ \hfill & {\displaystyle \times {\int }_0^1\cdots {\int }_0^1\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]d{x}_1\cdots d{x}_{p-1}dr}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle I_2=\frac{2a\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }{r}^{p+2N-3}exp\left(-q{r}^{2a}\right)}\hfill \\ \hfill & {\displaystyle \times {\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}log\left[{\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)\right]d{x}_1\cdots d{x}_{p-1}dr.}\hfill \end{array}$$

The $I_1$ can be calculated as

$$\begin{array}{cc}{\displaystyle I_1}\hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }\frac{t^{\frac{p+2N-2}{2a}-1}}{q^{\frac{2N+p-2}{2a}}}log{\left(\frac{t}{q}\right)}^{\frac{1}{a}}exp(-t)\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)dt}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{a{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\left\{a\frac{\partial }{\partial N}{\int }_0^{\infty }{t}^{\frac{p+2N-2}{2a}-1}exp(-t)-\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)logq\right\}\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{a{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}\left\{a\frac{\partial }{\partial N}\mathsf{\Gamma }\left(\frac{p+2N-2}{2a}\right)-\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)logq\right\}\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{a{\pi }^{\frac{p}{2}}}\left\{\psi \left(\frac{p+2N-2}{2a}\right)-logq\right\}\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right),}\hfill \end{array}$$

where $t=q{r}^{2a}$.

The $I_2$ can be calculated as

$$\begin{array}{cc}{\displaystyle I_2}\hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)q^{\frac{2N+p-2}{2a}}}{{\pi }^{\frac{p}{2}}\mathsf{\Gamma }\left(\frac{2N+p-2}{2a}\right)}{\int }_0^{\infty }\frac{t^{\frac{2N+p-2}{2a}-1}}{q^{\frac{2N+p-2}{2a}}}exp(-t)}\hfill \\ \hfill & {\displaystyle \times {\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}log\left[{\mathcal{B}}_p\left(x_1,\dots ,x_{p-1}\right)\right]d{x}_1\cdots d{x}_{p-1}dt}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times \left\{log{\lambda }_1+log\left[1+\frac{\left({\lambda }_2-{\lambda }_1\right)x_1}{{\lambda }_1}+\cdots +\frac{\left({\lambda }_p-{\lambda }_{p-1}\right)x_1{x}_2\cdots x_{p-1}}{{\lambda }_1}\right]\right\}d{x}_1\cdots d{x}_{p-1}}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}log{\lambda }_1\left[\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)\right]-\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}{\int }_0^1\cdots {\int }_0^1\left\{\prod _{j=1}^{p-1}\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]\right\}}\hfill \\ \hfill & {\displaystyle \times \sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}{\left[\frac{\left({\lambda }_2-{\lambda }_1\right)x_1}{{\lambda }_1}+\cdots +\frac{\left({\lambda }_p-{\lambda }_{p-1}\right)x_1{x}_2\cdots x_{p-1}}{{\lambda }_1}\right]}^{k}d{x}_1\cdots d{x}_{p-1}}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}log{\lambda }_1\left[\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)\right]-\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_{p-1}=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_{p-1}}\right)}\hfill \\ \hfill & {\displaystyle \times {\int }_0^1\cdots {\int }_0^1\prod _{j=1}^{p-1}\left\{\left[x_j^{\frac{p-j}{2}-1}{\left(1-x_j\right)}^{-\frac{1}{2}}\right]{\left[\frac{\left({\lambda }_{j+1}-{\lambda }_j\right)}{{\lambda }_1}\left(\prod _{\ell =1}^j{x}_{\ell }\right)\right]}^{i_j}\right\}d{x}_1\cdots d{x}_{p-1}}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}log{\lambda }_1\left[\prod _{j=1}^{p-1}B\left(\frac{p-j}{2},\frac{1}{2}\right)\right]-\frac{\mathsf{\Gamma }\left(\frac{p}{2}\right)}{{\pi }^{\frac{p}{2}}}\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_{p-1}=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_{p-1}}\right)}\hfill \\ \hfill & {\displaystyle \times \prod _{j=1}^{p-1}\left\{{\left[\frac{\left({\lambda }_{j+1}-{\lambda }_j\right)}{{\lambda }_1}\right]}^{i_j}B\left(\sum _{\ell =j}^{p-1}{i}_{\ell }+\frac{p-j}{2},\frac{1}{2}\right)\right\}}\hfill \end{array}$$

provided that the infinite sum converges.

Hence, the required.

4.7. Proof for Section 2.6

The corresponding KLD can be expressed as

$$\begin{array}{cc}{\displaystyle KLD}\hfill & {\displaystyle =log\left(\frac{a_1\cdots a_p}{b_1\cdots b_p}\right)+\sum _{\ell =1}^p\left(b_{\ell }-a_{\ell }\right)E\left(X_{\ell }\right)}\hfill \\ \hfill & {\displaystyle +(p+1)E\left\{log\left[1+exp\left(-b_1{X}_1\right)+\cdots +exp\left(-b_p{X}_p\right)\right]\right\}}\hfill \\ \hfill & {\displaystyle -(p+1)E\left\{log\left[1+exp\left(-a_1{X}_1\right)+\cdots +exp\left(-a_p{X}_p\right)\right]\right\}}\hfill \\ \hfill & =log\left({\displaystyle \frac{a_1\cdots a_p}{b_1\cdots b_p}}\right)+(p+1)E\left\{log\left[1+exp\left(-b_1{X}_1\right)+\cdots +exp\left(-b_p{X}_p\right)\right]\right\}\hfill \\ \hfill & {\displaystyle -(p+1)E\left\{log\left[1+exp\left(-a_1{X}_1\right)+\cdots +exp\left(-a_p{X}_p\right)\right]\right\}}\hfill \end{array}$$

(11)

since the expectations are zero. Using the Taylor expansion for $log(1+z)$, the first expectation in (11) can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle -\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}E\left\{{\left[exp\left(-b_1{X}_1\right)+\cdots +exp\left(-b_p{X}_p\right)\right]}^k\right\}}\hfill \\ \hfill & {\displaystyle =-\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_p=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_p}\right)E\left[exp\left(-i_1{b}_1{X}_1-\cdots -i_p{b}_p{X}_p\right)\right]}\hfill \\ \hfill & =-\sum _{k=1}^{\infty }{\displaystyle \frac{{(-1)}^k}{k}}\sum _{i_1+\cdots +i_p=k}\left({\displaystyle \genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_p}}\right)\left[\prod _{j=1}^p\mathsf{\Gamma }\left({\displaystyle \frac{a_j+i_j{b}_j}{a_j}}\right)\right]\mathsf{\Gamma }\left(1-\sum _{j=1}^p{\displaystyle \frac{i_j{b}_j}{a_j}}\right),\hfill \end{array}$$

where the last step follows by Lemma 1 provided that ${\sum }_{j=1}^p{\displaystyle \frac{i_j{b}_j}{a_j}}<1$ and the infinite series converges. The second expectation in (11) can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle p!a_1\cdots a_p{\int }_{{\mathbb{R}}^p}\left\{{\left.\frac{\partial }{\partial \alpha }{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{\alpha }\right|}_{\alpha =0}\right\}}\hfill \\ \hfill & {\displaystyle \times \frac{exp\left(-a_1{x}_1-\cdots -a_p{x}_p\right)d{x}_1\cdots d{x}_p}{{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{p+1}}}\hfill \\ \hfill & {\displaystyle =p!a_1\cdots a_p\frac{\partial }{\partial \alpha }{\left.\left\{{\int }_{{\mathbb{R}}^p}\frac{exp\left(-a_1{x}_1-\cdots -a_p{x}_p\right)}{{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{p+1-\alpha }}d{x}_1\cdots d{x}_p\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle =p!\frac{\partial }{\partial \alpha }{\left.\left[\frac{\mathsf{\Gamma }(1-\alpha )}{\mathsf{\Gamma }(p+1-\alpha )}\right]\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle =\frac{{\mathsf{\Gamma }}^{\prime }(p+1)-\mathsf{\Gamma }(p+1){\mathsf{\Gamma }}^{\prime }\left(1\right)}{p!},}\hfill \end{array}$$

where the penultimate step is followed by Lemma 1. Hence, the required.

4.8. Proof for Section 2.7

The corresponding KLD can be expressed as

$$\begin{array}{cc}{\displaystyle KLD}\hfill & {\displaystyle =log\left[\frac{{\left(b\right)}_p{a}_1\cdots a_p}{{\left(d\right)}_p{b}_1\cdots b_p}\right]+\sum _{\ell =1}^p\left(c_{\ell }-a_{\ell }\right)E\left(X_{\ell }\right)}\hfill \\ \hfill & {\displaystyle +(d+p)E\left\{log\left[1+exp\left(-c_1{X}_1\right)+\cdots +exp\left(-c_p{X}_p\right)\right]\right\}}\hfill \\ \hfill & {\displaystyle -(b+p)E\left\{log\left[1+exp\left(-a_1{X}_1\right)+\cdots +exp\left(-a_p{X}_p\right)\right]\right\}}\hfill \\ \hfill & {\displaystyle =log\left[\frac{{\left(b\right)}_p{a}_1\cdots a_p}{{\left(d\right)}_p{b}_1\cdots b_p}\right]+(d+p)E\left\{log\left[1+exp\left(-c_1{X}_1\right)+\cdots +exp\left(-c_p{X}_p\right)\right]\right\}}\hfill \\ \hfill & {\displaystyle -(b+p)E\left\{log\left[1+exp\left(-a_1{X}_1\right)+\cdots +exp\left(-a_p{X}_p\right)\right]\right\}}\hfill \end{array}$$

(12)

since the expectations are zero. Using the Taylor expansion for $log(1+z)$, the first expectation in (12) can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle -\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}E\left\{{\left[exp\left(-c_1{X}_1\right)+\cdots +exp\left(-c_p{X}_p\right)\right]}^k\right\}}\hfill \\ \hfill & {\displaystyle =-\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_p=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_p}\right)E\left[exp\left(-i_1{c}_1{X}_1-\cdots -i_p{c}_p{X}_p\right)\right]}\hfill \\ \hfill & {\displaystyle =-\frac{1}{\mathsf{\Gamma }\left(b\right)}\sum _{k=1}^{\infty }\frac{{(-1)}^k}{k}\sum _{i_1+\cdots +i_p=k}\left(\genfrac{}{}{0pt}{}{k}{i_1,\dots ,i_p}\right)\left[\prod _{j=1}^p\mathsf{\Gamma }\left(\frac{a_j+i_j{c}_j}{a_j}\right)\right]\mathsf{\Gamma }\left(b-\sum _{j=1}^p\frac{i_j{c}_j}{a_j}\right),}\hfill \end{array}$$

where the last step follows by Lemma 1 provided that ${\sum }_{j=1}^p{\displaystyle \frac{i_j{c}_j}{a_j}}<b$ and the infinite series converges. The second expectation in (12) can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle {\left(b\right)}_p{a}_1\cdots a_p{\int }_{{\mathbb{R}}^p}\left\{{\left.\frac{\partial }{\partial \alpha }{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{\alpha }\right|}_{\alpha =0}\right\}}\hfill \\ \hfill & {\displaystyle \times \frac{exp\left(-a_1{x}_1-\cdots -a_p{x}_p\right)d{x}_1\cdots d{x}_p}{{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{b+p}}}\hfill \\ \hfill & {\displaystyle ={\left(b\right)}_p{a}_1\cdots a_p\frac{\partial }{\partial \alpha }{\left.\left\{{\int }_{{\mathbb{R}}^p}\frac{exp\left(-a_1{x}_1-\cdots -a_p{x}_p\right)}{{\left[1+exp\left(-a_1{x}_1\right)+\cdots +exp\left(-a_p{x}_p\right)\right]}^{b+p-\alpha }}d{x}_1\cdots d{x}_p\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle ={\left(b\right)}_p\frac{\partial }{\partial \alpha }{\left.\left[\frac{\mathsf{\Gamma }(b-\alpha )}{\mathsf{\Gamma }(b+p-\alpha )}\right]\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle =\frac{\mathsf{\Gamma }\left(b\right){\mathsf{\Gamma }}^{\prime }(b+p)-\mathsf{\Gamma }(b+p){\mathsf{\Gamma }}^{\prime }\left(b\right)}{\mathsf{\Gamma }\left(b\right)\mathsf{\Gamma }(b+p)},}\hfill \end{array}$$

where the penultimate step is followed by Lemma 1. Hence, the required.

4.9. Proof for Section 2.8

The corresponding KLD can be expressed as

$$\begin{array}{cc}KLD\hfill & {\displaystyle =log\left[\frac{\sqrt{a_1\cdots a_p}{\beta }_p\left(c,a_1,\dots ,a_p\right)}{\sqrt{b_1\cdots b_p}{\beta }_p\left(d,b_1,\dots ,b_p\right)}\right]}\hfill \\ \hfill & {\displaystyle +\frac{1}{2}\sum _{i=1}^p\left(b_i-a_i\right)E\left(X_i^2\right)}\hfill \\ \hfill & {\displaystyle +\frac{1}{2}\left(d{b}_1\cdots b_p-c{a}_1\cdots a_p\right)E\left(X_1^2\cdots X_p^2\right).}\hfill \end{array}$$

(13)

Using results in [8], we can calculate (13) as the required.

4.10. Proof for Section 2.9

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{\mathsf{\Gamma }\left(\frac{K}{2}+a+b-1\right)\mathsf{\Gamma }\left(c\right)\mathsf{\Gamma }\left(\frac{K}{2}+d-1\right)}{\mathsf{\Gamma }\left(a\right)\mathsf{\Gamma }\left(\frac{K}{2}+b-1\right)\mathsf{\Gamma }\left(\frac{K}{2}+c+d-1\right)}\right]+(b-d)E\left[log\left(\sum _{i=1}^K{X}_i^2\right)\right]}\hfill \\ \hfill & {\displaystyle +(a-c)E\left[log\left(1-\sum _{i=1}^K{X}_i^2\right)\right].}\hfill \end{array}$$

(14)

It is easy to show that

$$\begin{array}{c}{\displaystyle E\left[log\left(\sum _{i=1}^K{X}_i^2\right)\right]=\psi \left(\frac{K}{2}+b-1\right)-\psi \left(\frac{K}{2}+a+b-1\right)}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left(1-\sum _{i=1}^K{X}_i^2\right)\right]=\psi \left(a\right)-\psi \left(\frac{K}{2}+a+b-1\right),}\hfill \end{array}$$

so (14) reduces to the required.

4.11. Proof for Section 2.10

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{C\left(d,e,f\right)}{C\left(a,b,c\right)}\right]+(a-d)E\left[log\left(\prod _{i=1}^p{X}_i\right)\right]+(b-e)E\left[log\left(\prod _{i=1}^p\left(1-X_i\right)\right)\right]}\hfill \\ \hfill & {\displaystyle +2(c-f)E\left[log\left(\prod _{1\le i<j\le p}\left(X_i-X_j\right)\right)\right].}\hfill \end{array}$$

(15)

Easy calculations show that

$$\begin{array}{c}{\displaystyle E\left[log\left(\prod _{i=1}^p{X}_i\right)\right]={\left.\frac{\partial }{\partial \alpha }\left[\frac{C(a,b,c)}{C(a+\alpha ,b,c)}\right]\right|}_{\alpha =0},}\hfill \end{array}$$

$$\begin{array}{c}{\displaystyle E\left[log\left(\prod _{i=1}^p\left(1-X_i\right)\right)\right]={\left.\frac{\partial }{\partial \alpha }\left[\frac{C(a,b,c)}{C(a,b+\alpha ,c)}\right]\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left(\prod _{1\le i<j\le p}\left(X_i-X_j\right)\right)\right]={\left.\frac{\partial }{\partial \alpha }\left[\frac{C(a,b,c)}{C(a,b,c+\alpha )}\right]\right|}_{\alpha =0},}\hfill \end{array}$$

so (7) reduces to the required.

4.12. Proof for Section 2.11

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\displaystyle b_{p+1}\left(\sum _{i=1}^p{a}_i\right)\left(\prod _{i=1}^p{a}_i\right)}}{{\displaystyle a_{p+1}\left(\sum _{i=1}^p{b}_i\right)\left(\prod _{i=1}^p{b}_i\right)}}\right]+\sum _{i=1}^p\left(b_i-a_i\right)E\left(X_i\right)}\hfill \\ \hfill & {\displaystyle +E\left[log\left\{1-exp\left[-a_{p+1}min\left(X_1,\dots ,X_p\right)\right]\right\}\right]}\hfill \\ \hfill & {\displaystyle -E\left[log\left\{1-exp\left[-b_{p+1}min\left(X_1,\dots ,X_p\right)\right]\right\}\right].}\hfill \end{array}$$

(16)

Using the series expansion for $log(1+z)$, we can express (16) as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\displaystyle b_{p+1}\left(\sum _{i=1}^p{a}_i\right)\left(\prod _{i=1}^p{a}_i\right)}}{{\displaystyle a_{p+1}\left(\sum _{i=1}^p{b}_i\right)\left(\prod _{i=1}^p{b}_i\right)}}\right]+\sum _{i=1}^p\left(b_i-a_i\right)E\left(X_i\right)}\hfill \\ \hfill & {\displaystyle -\sum _{k=1}^{\infty }\frac{E\left\{exp\left[-k{a}_{p+1}min\left(X_1,\dots ,X_p\right)\right]\right\}}{k}}\hfill \\ \hfill & {\displaystyle +\sum _{k=1}^{\infty }\frac{E\left\{exp\left[-k{b}_{p+1}min\left(X_1,\dots ,X_p\right)\right]\right\}}{k}.}\hfill \end{array}$$

Hence, the required.

4.13. Proof for Section 2.12

The corresponding KLD can be expressed as

$$\begin{array}{cc}{\displaystyle KLD}\hfill & {\displaystyle =log\left[\frac{{\kappa }_1^{\frac{p}{2}-1}}{{\kappa }_2^{\frac{p}{2}-1}}\frac{I_{\frac{p}{2}-1}\left({\kappa }_2\right)}{I_{\frac{p}{2}-1}\left({\kappa }_1\right)}\right]+\left({\kappa }_1{\mathit{\mu }}_1^T-{\kappa }_2{\mathit{\mu }}_2^T\right)E\left(\mathbf{X}\right)}\hfill \\ \hfill & {\displaystyle =log\left[\frac{{\kappa }_1^{\frac{p}{2}-1}}{{\kappa }_2^{\frac{p}{2}-1}}\frac{I_{\frac{p}{2}-1}\left({\kappa }_2\right)}{I_{\frac{p}{2}-1}\left({\kappa }_1\right)}\right]+\left({\kappa }_1{\mathit{\mu }}_1^T-{\kappa }_2{\mathit{\mu }}_2^T\right){\mathit{\mu }}_1.}\hfill \end{array}$$

Hence, the required.

4.14. Proof for Section 3.1

The corresponding KLD can be expressed as

$$\begin{array}{c}{\displaystyle KLD=log\left[\frac{{\left|\mathbf{\Omega }\right|}^{c+d-a-b}{B}_p(c,d)}{B_p(a,b)}\right]+(a-c)E\left[log\left|\mathbf{\Omega }-\mathbf{X}\right|\right]+(b-d)E\left[log\left|\mathbf{X}\right|\right].}\hfill \end{array}$$

(17)

The expectations in (17) can be calculated as

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{\Omega }-\mathbf{X}\right|\right]={\left.{\displaystyle \frac{\partial }{\partial \alpha }\left\{\int \frac{{\left|\mathbf{\Omega }-\mathbf{x}\right|}^{\alpha +a-\frac{p+1}{2}}{\left|\mathbf{x}\right|}^{b-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{a+b}{B}_p(a,b)}d\mathbf{x}\right\}}\right|}_{\alpha =0}={\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(\alpha +a,b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\left\{\int \frac{{\left|\mathbf{\Omega }-\mathbf{x}\right|}^{a-\frac{p+1}{2}}{\left|\mathbf{x}\right|}^{\alpha +b-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{a+b}{B}_p(a,b)}d\mathbf{x}\right\}\right|}_{\alpha =0}={\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(a,\alpha +b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}.}\hfill \end{array}$$

Hence, the required.

4.15. Proof for Section 3.2

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{B_p\left(b_1,\dots ,b_n;b_{n+1}\right)}{B_p\left(a_1,\dots ,a_n;a_{n+1}\right)}\right]+\left(a_{n+1}-b_{n+1}\right)E\left[log\left|{\mathbf{I}}_p-\sum _{i=1}^n{\mathbf{X}}_i\right|\right]}\hfill \\ \hfill & {\displaystyle +\sum _{i=1}^n\left(a_i-b_i\right)E\left[log\left|{\mathbf{X}}_i\right|\right].}\hfill \end{array}$$

(18)

It is easy to show that

$$\begin{array}{c}{\displaystyle E\left[log\left|{\mathbf{X}}_i\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{B_p\left(a_1,\dots ,a_i+\alpha ,\dots ,a_n;a_{n+1}\right)}{B_p\left(a_1,\dots ,a_i,\dots ,a_n;a_{n+1}\right)}\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left|{\mathbf{I}}_p-\sum _{i=1}^n{\mathbf{X}}_i\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{B_p\left(a_1,\dots ,a_n;a_{n+1}+\alpha \right)}{B_p\left(a_1,\dots ,a_n;a_{n+1}\right)}\right|}_{\alpha =0},}\hfill \end{array}$$

so (18) reduces to the required.

4.16. Proof for Section 3.3

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{d^{pc}{\mathsf{\Gamma }}_p\left(c\right)}{b^{pa}{\mathsf{\Gamma }}_p\left(a\right)}\frac{{\left|{\mathbf{\Sigma }}_2\right|}^c}{{\left|{\mathbf{\Sigma }}_1\right|}^a}\right]+(a-c)E\left[log\left|\mathbf{X}\right|\right]+\mathrm{tr}\left[-\frac{1}{b}{\mathbf{\Sigma }}_1^{-1}E\left(\mathbf{X}\right)\right]}\hfill \\ \hfill & {\displaystyle -\mathrm{tr}\left[-\frac{1}{d}{\mathbf{\Sigma }}_2^{-1}E\left(\mathbf{X}\right)\right].}\hfill \end{array}$$

(19)

The first expectation in (19) can be calculated as

$$\begin{array}{cc}{\displaystyle E\left[log\left|\mathbf{X}\right|\right]}\hfill & {\displaystyle =\frac{{\left|{\mathbf{\Sigma }}_1\right|}^{-a}}{b^{ap}{\mathsf{\Gamma }}_p\left(a\right)}{\left.\frac{\partial }{\partial \alpha }\left\{\int {\left|\mathbf{x}\right|}^{\alpha +a-\frac{p+1}{2}}exp\left[\mathrm{tr}\left(-\frac{1}{b}{\mathbf{\Sigma }}_1^{-1}\mathbf{x}\right)\right]\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle =\frac{1}{{\mathsf{\Gamma }}_p\left(a\right)}{\left.\frac{\partial }{\partial \alpha }\left[{\left|{\mathbf{\Sigma }}_1\right|}^{\alpha }{b}^{p\alpha }{\mathsf{\Gamma }}_p(a+\alpha )\right]\right|}_{\alpha =0}.}\hfill \end{array}$$

Since $E\left(\mathbf{X}\right)=2a{\mathbf{\Sigma }}_1$, the second and third terms in (19) are equal to

$$\begin{array}{c}{\displaystyle \mathrm{tr}\left[-\frac{1}{b}{\mathbf{\Sigma }}_1^{-1}E\left(\mathbf{X}\right)\right]=\frac{2ap}{b}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle \mathrm{tr}\left[-\frac{1}{d}{\mathbf{\Sigma }}_2^{-1}E\left(\mathbf{X}\right)\right]=\mathrm{tr}\left[-\frac{2a}{d}{\mathbf{\Sigma }}_2^{-1}{\mathbf{\Sigma }}_1\right],}\hfill \end{array}$$

respectively. Hence, the required.

4.17. Proof for Section 3.4

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{B_p(d,e){ }_2{F}_1\left(d,f;d+e;-\mathbf{B}\right)}{B_p(a,b){ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right]+(a-d)E\left[log\left|\mathbf{X}\right|\right]+(b-e)E\left[log\left|{\mathbf{I}}_p-\mathbf{X}\right|\right]}\hfill \\ \hfill & {\displaystyle +(f-c)E\left[log\left|{\mathbf{I}}_p+\mathbf{B}\mathbf{X}\right|\right].}\hfill \end{array}$$

(20)

The expectations in (20) can be easily calculated as

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{B_p(a+\alpha ,b){ }_2{F}_1\left(a+\alpha ,c;a+b+\alpha ;-\mathbf{B}\right)}{B_p(a,b){ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right|}_{\alpha =0},}\hfill \end{array}$$

$$\begin{array}{c}{\displaystyle E\left[log\left|{\mathbf{I}}_p-\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{B_p(a,b+\alpha ){ }_2{F}_1\left(a,c;a+b+\alpha ;-\mathbf{B}\right)}{B_p(a,b){ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left|{\mathbf{I}}_p+\mathbf{B}\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{{ }_2{F}_1\left(a,c-\alpha ;a+b;-\mathbf{B}\right)}{{ }_2{F}_1\left(a,c;a+b;-\mathbf{B}\right)}\right|}_{\alpha =0}.}\hfill \end{array}$$

Hence, the required.

4.18. Proof for Section 3.5

The corresponding KLD can be expressed as

$$\begin{array}{c}{\displaystyle KLD=log\left[\frac{{\left|\mathbf{\Omega }\right|}^{a-c}{B}_p(c,d)}{B_p(a,b)}\right]+(c+d-a-b)E\left[log\left|\mathbf{\Omega }+\mathbf{X}\right|\right]+(b-d)E\left[log\left|\mathbf{X}\right|\right].}\hfill \end{array}$$

(21)

The expectations in (21) can be calculated as

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{\Omega }+\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\left\{\int \frac{{\left|\mathbf{\Omega }+\mathbf{x}\right|}^{\alpha -a-b}{\left|\mathbf{x}\right|}^{b-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{-a}{B}_p(a,b)}d\mathbf{x}\right\}\right|}_{\alpha =0}={\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(a-\alpha ,b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\left\{\int \frac{{\left|\mathbf{\Omega }+\mathbf{x}\right|}^{-a-b}{\left|\mathbf{x}\right|}^{\alpha +b-\frac{p+1}{2}}}{{\left|\mathbf{\Omega }\right|}^{-a}{B}_p(a,b)}d\mathbf{x}\right\}\right|}_{\alpha =0}={\left.\frac{\partial }{\partial \alpha }\left\{\frac{{\left|\mathbf{\Omega }\right|}^{\alpha }{B}_p(a-\alpha ,\alpha +b)}{B_p(a,b)}\right\}\right|}_{\alpha =0}.}\hfill \end{array}$$

Hence, the required.

4.19. Proof for Section 3.6

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{d^{pc}{\mathsf{\Gamma }}_p\left(c\right)}{b^{pa}{\mathsf{\Gamma }}_p\left(a\right)}\frac{{\left|{\mathbf{\Sigma }}_1\right|}^a}{{\left|{\mathbf{\Sigma }}_2\right|}^c}\right]+(c-a)E\left[log\left|\mathbf{X}\right|\right]+\frac{1}{d}\mathrm{tr}\left[{\mathbf{\Sigma }}_{2}E\left({\mathbf{X}}^{-1}\right)\right]}\hfill \\ \hfill & {\displaystyle -\frac{1}{b}\mathrm{tr}\left[{\mathbf{\Sigma }}_{1}E\left({\mathbf{X}}^{-1}\right)\right].}\hfill \end{array}$$

(22)

The first expectation in (22) can be calculated as

$$\begin{array}{cc}{\displaystyle E\left[log\left|\mathbf{X}\right|\right]}\hfill & {\displaystyle =\frac{{\left|{\mathbf{\Sigma }}_1\right|}^a}{b^{ap}{\mathsf{\Gamma }}_p\left(a\right)}{\left.\frac{\partial }{\partial \alpha }\left\{\int {\left|\mathbf{x}\right|}^{\alpha -a-\frac{p+1}{2}}exp\left[\mathrm{tr}\left(-\frac{1}{b}{\mathbf{\Sigma }}_1\mathbf{x}\right)\right]\right\}\right|}_{\alpha =0}}\hfill \\ \hfill & {\displaystyle =\frac{1}{{\mathsf{\Gamma }}_p\left(a\right)}{\left.\frac{\partial }{\partial \alpha }\left[\frac{{\left|{\mathbf{\Sigma }}_1\right|}^{\alpha }{\mathsf{\Gamma }}_p(a-\alpha )}{b^{p\alpha }}\right]\right|}_{\alpha =0}.}\hfill \end{array}$$

Since $E\left({\mathbf{X}}^{-1}\right)=2a{\mathbf{\Sigma }}_1^{-1}$, the second and third terms in (22) are equal to

$$\begin{array}{c}{\displaystyle \mathrm{tr}\left[{\mathbf{\Sigma }}_{2}E\left({\mathbf{X}}^{-1}\right)\right]=2a\mathrm{tr}\left[{\mathbf{\Sigma }}_2{\mathbf{\Sigma }}_1^{-1}\right]}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle \mathrm{tr}\left[{\mathbf{\Sigma }}_{1}E\left({\mathbf{X}}^{-1}\right)\right]=2ap,}\hfill \end{array}$$

respectively. Hence, the required.

4.20. Proof for Section 3.7

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{B_p(c,d){ }_1{F}_1\left(c;c+d;-{\mathbf{B}}_2\right)}{B_p(a,b){ }_1{F}_1\left(a,a+b;-{\mathbf{B}}_1\right)}\right]+(a-c)E\left[log\left|\mathbf{X}\right|\right]+(b-d)E\left[log\left|{\mathbf{I}}_p-\mathbf{X}\right|\right]}\hfill \\ \hfill & {\displaystyle +\mathrm{tr}\left[\left({\mathbf{B}}_2-{\mathbf{B}}_1\right)E\left(\mathbf{X}\right)\right].}\hfill \end{array}$$

(23)

The expectations in (23) can be easily calculated as

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{B_p(a+\alpha ,b){ }_1{F}_1\left(a+\alpha ;a+b+\alpha ;-{\mathbf{B}}_1\right)}{B_p(a,b){ }_1{F}_1\left(a;a+b;-{\mathbf{B}}_1\right)}\right|}_{\alpha =0},}\hfill \end{array}$$

$$\begin{array}{c}{\displaystyle E\left[log\left|{\mathbf{I}}_p-\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{B_p(a,b+\alpha ){ }_1{F}_1\left(a;a+b+\alpha ;-{\mathbf{B}}_1\right)}{B_p(a,b){ }_1{F}_1\left(a;a+b;-{\mathbf{B}}_1\right)}\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left(\mathbf{X}\right)={\left.\frac{\partial }{\partial \mathbf{z}}\frac{{ }_1{F}_1\left(a;a+b;\mathbf{z}-{\mathbf{B}}_1\right)}{{ }_1{F}_1\left(a;a+b;-{\mathbf{B}}_1\right)}\right|}_{\mathbf{z}=\mathbf{0}}.}\hfill \end{array}$$

Hence, the required.

4.21. Proof for Section 3.8

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\mathsf{\Gamma }}_p\left(c\right){\mathsf{\Psi }}_1\left(c;c-d+\frac{p+1}{2};{\mathbf{B}}_2\right)}{{\mathsf{\Gamma }}_p\left(a\right){\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right]+(a-c)E\left[log\left|\mathbf{X}\right|\right]+(d-b)E\left[log\left|{\mathbf{I}}_p+\mathbf{X}\right|\right]}\hfill \\ \hfill & {\displaystyle +\mathrm{tr}\left[\left({\mathbf{B}}_2-{\mathbf{B}}_1\right)E\left(\mathbf{X}\right)\right].}\hfill \end{array}$$

(24)

The expectations in (24) can be easily calculated as

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{{\mathsf{\Gamma }}_p(a+\alpha ){\mathsf{\Psi }}_1\left(a+\alpha ;a-b+\alpha +\frac{p+1}{2};{\mathbf{B}}_1\right)}{{\mathsf{\Gamma }}_p\left(a\right){\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right|}_{\alpha =0},}\hfill \end{array}$$

$$\begin{array}{c}{\displaystyle E\left[log\left|{\mathbf{I}}_p+\mathbf{X}\right|\right]={\left.\frac{\partial }{\partial \alpha }\frac{{\mathsf{\Psi }}_1\left(a;a-b+\alpha +\frac{p+1}{2};{\mathbf{B}}_1\right)}{{\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left(\mathbf{X}\right)={\left.\frac{\partial }{\partial \mathbf{z}}\frac{{\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1-\mathbf{z}\right)}{{\mathsf{\Psi }}_1\left(a;a-b+\frac{p+1}{2};{\mathbf{B}}_1\right)}\right|}_{\mathbf{z}=\mathbf{0}}.}\hfill \end{array}$$

Hence, the required.

4.22. Proof for Section 3.9

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{{\left|{\mathbf{V}}_2\right|}^{\frac{n}{2}}{\left|{\mathbf{U}}_2\right|}^{\frac{p}{2}}}{{\left|{\mathbf{V}}_1\right|}^{\frac{n}{2}}{\left|{\mathbf{U}}_1\right|}^{\frac{p}{2}}}\right]+\frac{1}{2}E\left\{\mathrm{tr}\left[{\mathbf{V}}_2^{-1}{\left(\mathbf{X}-{\mathbf{M}}_2\right)}^T{\mathbf{U}}_2^{-1}\left(\mathbf{X}-{\mathbf{M}}_2\right)\right]\right\}}\hfill \\ \hfill & {\displaystyle -\frac{1}{2}E\left\{\mathrm{tr}\left[{\mathbf{V}}_1^{-1}{\left(\mathbf{X}-{\mathbf{M}}_1\right)}^T{\mathbf{U}}_1^{-1}\left(\mathbf{X}-{\mathbf{M}}_1\right)\right]\right\}.}\hfill \end{array}$$

(25)

The second expectation in (25) can be expressed as

$$\begin{array}{c}{\displaystyle \mathrm{tr}\left\{{\mathbf{V}}_1^{-1}E\left[{\left(\mathbf{X}-{\mathbf{M}}_1\right)}^T\left(\mathbf{X}-{\mathbf{M}}_1\right)\right]{\mathbf{U}}_1^{-1}\right\}=\mathrm{tr}\left({\mathbf{U}}_1\right)\mathrm{tr}\left({\mathbf{U}}_1^{-1}\right).}\hfill \end{array}$$

The first expectation in (25) can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle \mathrm{tr}\left\{{\mathbf{V}}_2^{-1}E\left[{\left(\mathbf{X}-{\mathbf{M}}_2\right)}^T\left(\mathbf{X}-{\mathbf{M}}_2\right)\right]{\mathbf{U}}_2^{-1}\right\}}\hfill \\ \hfill & {\displaystyle =\mathrm{tr}\left[{\mathbf{V}}_2^{-1}E\left({\mathbf{X}}^T\mathbf{X}-{\mathbf{X}}^T{\mathbf{M}}_2-{\mathbf{M}}_2^T\mathbf{X}+{\mathbf{M}}_2^T{\mathbf{M}}_2\right){\mathbf{U}}_2^{-1}\right]}\hfill \\ \hfill & {\displaystyle =\mathrm{tr}\left\{{\mathbf{V}}_2^{-1}\left[E\left({\mathbf{X}}^T\mathbf{X}\right)-E\left({\mathbf{X}}^T{\mathbf{M}}_2\right)-E\left({\mathbf{M}}_2^T\mathbf{X}\right)+{\mathbf{M}}_2^T{\mathbf{M}}_2\right]{\mathbf{U}}_2^{-1}\right\}}\hfill \\ \hfill & {\displaystyle =\mathrm{tr}\left({\mathbf{U}}_1\right)\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{V}}_1{\mathbf{U}}_2^{-1}\right)+\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_1^T{\mathbf{M}}_1{\mathbf{U}}_2^{-1}\right)-\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_1^T{\mathbf{M}}_2{\mathbf{U}}_2^{-1}\right)}\hfill \\ \hfill & {\displaystyle -\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_2^T{\mathbf{M}}_1{\mathbf{U}}_2^{-1}\right)+\mathrm{tr}\left({\mathbf{V}}_2^{-1}{\mathbf{M}}_2^T{\mathbf{M}}_2{\mathbf{U}}_2^{-1}\right).}\hfill \end{array}$$

Hence, the required.

4.23. Proof for Section 3.10

The corresponding KLD can be expressed as

$$\begin{array}{cc}\hfill & {\displaystyle KLD=log\left[\frac{C\left(a\right)}{C\left(b\right)}\right]+(a-b)E\left[log\left|\mathbf{X}\right|I\left\{{\mathbf{0}}_p<\mathbf{X}\le \mathbf{B}\right\}\right]}\hfill \\ \hfill & {\displaystyle +(a-b)E\left[log\left|{\mathbf{I}}_p-\mathbf{X}\right|I\left\{\mathbf{B}<\mathbf{X}\le {\mathbf{I}}_p\right\}\right]}\hfill \\ \hfill & {\displaystyle +(b-a){\left|\mathbf{B}\right|}^{\frac{p+1}{2}}log\left|\mathbf{B}\right|B_p\left(a,\frac{p+1}{2}\right)}\hfill \\ \hfill & {\displaystyle +(b-a){\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{\frac{p+1}{2}}log\left|{\mathbf{I}}_p-\mathbf{B}\right|B_p\left(\frac{p+1}{2},a\right).}\hfill \end{array}$$

(26)

The expectations in (26) can be easily calculated as

$$\begin{array}{c}{\displaystyle E\left[log\left|\mathbf{X}\right|I\left\{{\mathbf{0}}_p<\mathbf{X}\le \mathbf{B}\right\}\right]={\left.\frac{\partial }{\partial \alpha }\left\{{\left|\mathbf{B}\right|}^{\alpha +\frac{p+1}{2}}{B}_p\left(a+\alpha ,\frac{p+1}{2}\right)\right\}\right|}_{\alpha =0}}\hfill \end{array}$$

and

$$\begin{array}{c}{\displaystyle E\left[log\left|{\mathbf{I}}_p-\mathbf{X}\right|I\left\{\mathbf{B}<\mathbf{X}\le {\mathbf{I}}_p\right\}\right]={\left.\frac{\partial }{\partial \alpha }\left\{{\left|{\mathbf{I}}_p-\mathbf{B}\right|}^{\alpha +\frac{p+1}{2}}{B}_p\left(\frac{p+1}{2},a+\alpha \right)\right\}\right|}_{\alpha =0}.}\hfill \end{array}$$

Hence, the required.