**7. Particular Cases: Univariate and Bivariate Cauchy Distribution**

*7.1. Case of p* = 1

This case corresponds to the univariate Cauchy distribution. The KLD is given by

$$\text{KL}(X^1||X^2) = -\frac{1}{2}\log\lambda - \frac{\partial}{\partial a}\left\{ \,\_2F\_1(a, \frac{1}{2}; a+1; 1-\lambda) \right\}\Big|\_{a=0} \tag{119}$$

where <sup>2</sup>*F*<sup>1</sup> is the Gauss's hypergeometric function. The expression of the derivative of <sup>2</sup>*F*<sup>1</sup> is given as follows (see Appendix C.1 for details of computation)

$$\begin{split} \frac{\partial}{\partial a} \left\{ \,\_2F\_1(a, \frac{1}{2}; a+1; 1-\lambda) \right\} \Big|\_{a=0} &= \sum\_{n=1}^{\infty} \left( \frac{1}{2} \right)\_n \frac{1}{n} \frac{(1-\lambda)^n}{n!} \\ &= -2 \ln \left( \frac{1+\lambda^{1/2}}{2} \right). \end{split} \tag{120}$$

Accordingly, the KLD is then expressed as

$$\text{KL}(X^1||X^2) = \log \frac{(1+\lambda^{\frac{1}{2}})^2}{4\lambda^{\frac{1}{2}}} \tag{121}$$

$$\hat{\lambda} = \log \frac{(1 + \lambda^{-\frac{1}{2}})^2}{4\lambda^{-\frac{1}{2}}} = \text{KL}(X^2 || X^1). \tag{122}$$

We conclude that KLD between Cauchy densities is always symmetric. Interestingly, this is consistent with the result presented in [31].

*7.2. Case of p* = 2

This case corresponds to the Bivariate Cauchy distribution. The KLD is then given by

$$\text{KL}(\mathbf{X}^1||\mathbf{X}^2) = -\frac{1}{2}\log\lambda\_1\lambda\_2 - \frac{3}{2}\frac{\partial}{\partial a}\left\{F\_1(a, \frac{1}{2}, \frac{1}{2}; a + \frac{3}{2}; 1 - \lambda\_1, 1 - \lambda\_2)\right\}\Big|\_{a=0} \tag{123}$$

where *F*<sup>1</sup> is the Appell's hypergeometric function (see Appendix A). The expression of the derivative of *F*<sup>1</sup> can be further developed

$$\begin{split} & \frac{\partial}{\partial a} \left\{ F\_1(a, \frac{1}{2}, \frac{1}{2}; a + \frac{3}{2}; 1 - \lambda\_1, 1 - \lambda\_2) \right\} \bigg|\_{a=0} \\ &= \sum\_{n,m=0}^{+\infty} \frac{(1)\_{m+n} (\frac{1}{2})\_n (\frac{1}{2})\_m}{(\frac{3}{2})\_{m+n}} \frac{1}{m+n} \frac{(1 - \lambda\_1)^n}{n!} \frac{(1 - \lambda\_2)^m}{m!} . \end{split} \tag{124}$$

In addition, when the eigenvalue *λ<sup>i</sup>* for *i* = 1, 2 takes some particular values, the expression of the KLD becomes very simple. In the following, we show some cases:

$$(\lambda\_1 = 1, \lambda\_2 \neq 1) \text{ or } (\lambda\_2 = 1, \lambda\_1 \neq 1)$$

For this particular case, we have

$$\left. \frac{\partial}{\partial a} \left\{ F\_1(a, \frac{1}{2}, \frac{1}{2}; a + \frac{3}{2}; 1 - \lambda\_i, 0) \right\} \right|\_{a=0} = \frac{\partial}{\partial a} \left\{ \,\_2F\_1(a, \frac{1}{2}; a + \frac{3}{2}; 1 - \lambda\_i) \right\} \Big|\_{a=0} \tag{125}$$

$$=-\ln\lambda\_i + \frac{1}{\sqrt{1-\lambda\_i}}\ln\left(\frac{1-\sqrt{1-\lambda\_i}}{1+\sqrt{1-\lambda\_i}}\right) + 2.\tag{126}$$

The demonstration of the derivation is shown in Appendix C.2. Then, KLD becomes equal to

$$\text{KL}(\mathbf{X}^1||\mathbf{X}^2) = \ln \lambda\_i - \frac{3}{2} \frac{1}{\sqrt{1-\lambda\_i}} \ln \left(\frac{1-\sqrt{1-\lambda\_i}}{1+\sqrt{1-\lambda\_i}}\right) - 3. \tag{127}$$

*λ*<sup>1</sup> = *λ*<sup>2</sup> = *λ*

For this particular case, we have

$$\left. \frac{\partial}{\partial a} \left\{ F\_1(a, \frac{1}{2}, \frac{1}{2}; a + \frac{3}{2}; 1 - \lambda, 1 - \lambda) \right\} \right|\_{a = 0} = \left. \frac{\partial}{\partial a} \left\{ \,\_2F\_1(a, 1; a + \frac{3}{2}; 1 - \lambda) \right\} \right|\_{a = 0} \tag{128}$$

$$=-\frac{2}{\sqrt{1-\lambda^{-1}}}\ln(\sqrt{\lambda}+\sqrt{\lambda-1})+2.\tag{129}$$

For more details about the demonstration see Appendix C.3. The KLD becomes equal to

$$\text{KL}(\mathbf{X}^1||\mathbf{X}^2) = -\ln \lambda + \frac{3}{\sqrt{1-\lambda^{-1}}} \ln(\sqrt{\lambda} + \sqrt{\lambda - 1}) - 3. \tag{130}$$

It is easy to deduce that

$$\text{KL}(\mathbf{X}^2||\mathbf{X}^1) = \ln \lambda + \frac{3}{\sqrt{1-\lambda}} \ln(\sqrt{\lambda^{-1}} + \sqrt{\lambda^{-1}-1}) - 3. \tag{131}$$

This result can be demonstrated using the same process as KL(**X**1||**X**2). It is worth to notice that KL(**X**1||**X**2) <sup>=</sup> KL(**X**2||**X**1) which leads us to conclude that the property of symmetry observed for the univariate case is no longer valid in the multivariate case. Nielsen et al. in [32] gave the same conclusion.

#### **8. Implementation and Comparison with Monte Carlo Technique**

In this section, we show how we practically compute the numerical values of the KLD, especially when we have several equivalent expressions which differ in the region of convergence. To reach this goal, the eigenvalues of **Σ**1**Σ**−<sup>1</sup> <sup>2</sup> are rearranged in a descending order *λ<sup>p</sup>* > *λp*−<sup>1</sup> > ... > *λ*<sup>1</sup> > 0. This operation is justified by Equation (53) where it can be seen that the permutation of the eigenvalues does not affect the expectation result. Three cases can be identified from the expressions of KLD.

*8.1. Case* 1 > *λ<sup>p</sup>* > *λp*−<sup>1</sup> > ... > *λ*<sup>1</sup> > 0

The expression of KL(**X**1||**X**2) is given by Equation (109) and KL(**X**2||**X**1) is given by (115).

$$\begin{aligned} \text{8.2. Case } \lambda\_p > \lambda\_{p-1} > \dots > \lambda\_1 > 1\\ \text{KL}(\mathbf{X}^1||\mathbf{X}^2) \text{ is given by the Equation (110) and KL}(\mathbf{X}^2||\mathbf{X}^1) \text{ is given by (114).} \end{aligned}$$
