Intrinsic Losses Based on Information Geometry and Their Applications

Abstract

One main interest of information geometry is to study the properties of statistical models that do not depend on the coordinate systems or model parametrization; thus, it may serve as an analytic tool for intrinsic inference in statistics. In this paper, under the framework of Riemannian geometry and dual geometry, we revisit two commonly-used intrinsic losses which are respectively given by the squared Rao distance and the symmetrized Kullback–Leibler divergence (or Jeffreys divergence). For an exponential family endowed with the Fisher metric and $\alpha$-connections, the two loss functions are uniformly described as the energy difference along an $\alpha$-geodesic path, for some $\alpha \in \{-1,0,1\}$. Subsequently, the two intrinsic losses are utilized to develop Bayesian analyses of covariance matrix estimation and range-spread target detection. We provide an intrinsically unbiased covariance estimator, which is verified to be asymptotically efficient in terms of the intrinsic mean square error. The decision rules deduced by the intrinsic Bayesian criterion provide a geometrical justification for the constant false alarm rate detector based on generalized likelihood ratio principle.

1. Introduction

In Bayesian analysis, the choices of a particular loss function and a featured type of priors strongly influence the resulting inference. From the standpoint of purely inference, the final results are typically required to be independent of the way that the model has been parameterized. Thus, in order to produce a Bayes estimator or action which only depends on the model assumed and the data observed, an intrinsic loss and a noninformative (or uninformative) prior are demanded. In an early paper [1], the intrinsic losses are also called the noninformative losses, since they are considered as an analogous determination of the noninformative priors. The reference [1] describes an intrinsic loss as the automated derivation from the sampling distribution without subjective inputs. Whereafter, the entropy and Hellinger losses are suggested there as two appropriate distances between distributions, and their properties are studied especially for exponential families. Some latter works by Barcelona et al. regard the additional requirements as reasonable that the intrinsic losses need be invariant under reduction to sufficient statistics [2]. A symmetric version of Kullback–Leibler (KL) divergence (also named “the intrinsic discrepancy”) as such an intrinsic loss is highly recommended, and has been widely applied to develop intrinsic Bayesian analyses of hypothesis testing [3,4], point estimation [4,5,6,7], and interval estimation [2,4,7].

The invariance criteria achieved by the intrinsic losses are indeed consistent with one main topic of information geometry, which concentrates on excavating the properties of statistical manifolds that are preserved over coordinate system transformations [8]. Under certain regularity conditions, each parametric statistical model possesses a Riemannian manifold structure given by the Fisher metric, with the parametrization identical to a coordinate system [9]. The parameters are merely labels for probability distributions, just as the coordinates for geometric objects. Moreover, the manifold structure supplied by the Fisher metric and $\alpha$-connections are proven to be invariant under transformations of coordinates and reduction by sufficiency [10]. Following this line, a concept is intrinsic if it has a well-defined geometrical meaning. The Riemannian distance in this case—also named the Rao distance [11]—is taken as an intrinsic measure for the dissimilarity between two probability distributions. The reference [12] regards the squared Rao distance between the models labelled by the actual value of the parameter and its estimate as the most natural intrinsic loss, which has been utilized to seek intrinsic Bayes estimators in some concrete univariate examples. In the wider literature, statisticians also apply the Rao distance measure to stimulate the intrinsic study of nonlinear filtering [13], estimation fusion [14,15], and target tracking [16]. Besides, the intrinsic versions of the bias, mean square error (MSE) and Cramér–Rao bound (CRB) are put forward [17,18], which help conduct a systematic analysis for intrinsic estimation theory.

From the above, we can conclude that the Rao distance and the KL divergence are widely accepted as foundation quantities for the intrinsic losses. The former is well-defined on the statistical manifolds with consideration of its usual Riemannian structure, while the later is extremely famous in information theory and is also associated with the canonical divergence in Amari’s dual geometries [9,19]. In this paper, we revisit two intrinsic losses that are given by the squared Rao distance and the symmetric KL divergence (also called the Jeffreys divergence), by integrating them in the framework of information geometry. For the exponential families, they are shown to be uniformly described as the energy difference along an $\alpha$-geodesic path on the statistical manifold, with some $\alpha \in \{-1,0,1\}$. Additionally, the Jeffreys prior—as one of the most popular noninformative priors—corresponds to the Riemannian volume in this setting [20]. The elucidation of their geometrical meanings contributes to deepen our understanding and capabilities in intrinsic Bayesian analysis. Subsequently, we apply these two intrinsic losses to develop Bayesian approaches to covariance matrix estimation and range-spread target detection, which are both hot issues in radar signal processing. The sample covariance matrix (the maximum likelihood estimator (MLE) using a set of zero-mean Gaussian samples) is proven to be intrinsically biased [18]. We provide a Bayesian approach to estimate the scale factor of the sample covariance matrix, which leads to an intrinsically unbiased and asymptotically efficient covariance estimator. The detection of range-spread target is, in essence, a decision problem. Intrinsic analysis supplies a novel way to derive a consistent decision rule under different coordinate systems of measurement. The produced detectors are equivalent to a classical result based on generalized likelihood ratio principle, which will benefit an intuitive interpretation about the geometrical nature of the latter.

In the next section, we review some basic facts about information geometry and exponential family. In Section 3, we present the intrinsic Bayesian approaches to point estimation and hypothesis testing. In Section 4, some applications to covariance matrix estimation and range-spread target detection are studied in detail. Section 5 concludes.

2. The Riemannian Geometry and Dual Geometry of Exponential Family

2.1. The Fisher Metric and the $\alpha$-Connections

Assume a statistical model $S=\{p(\mathit{x}|\mathit{\xi }):\mathit{\xi }\in \mathbf{\Xi }\}$ to describe the probabilistic behavior of a random variable $\mathit{x}$, with the parameter space $\mathbf{\Xi }$ being an open set of ${\mathbb{R}}^n$. Under certain regularity conditions, S can be regarded as a differentiable manifold, with $\mathit{\xi }=[{\xi }^1,\dots ,{\xi }^n]$ being a coordinate system. The manifold S carries the Riemannian structure induced by the Fisher information matrix $\mathit{G}(\mathit{\xi })=[g_{ij}(\mathit{\xi })]$,

$$g_{ij}(\mathit{\xi })={\mathbb{E}}_{\mathit{\xi }}\left[\frac{\partial logp(\mathit{x}|\mathit{\xi })}{\partial {\xi }^i}\frac{\partial logp(\mathit{x}|\mathit{\xi })}{\partial {\xi }^j}\right],$$

(1)

where ${\mathbb{E}}_{\mathit{\xi }}$ signifies the expectation with respect to $p(\mathit{x}|\mathit{\xi })$. In this geometric setting, $\mathit{G}$ plays the key role of a metric tensor [11], called the Fisher metric. Then, the infinitesimal squared distance between two closely-spaced distributions $p(\mathit{x}|\mathit{\xi })$ and $p(\mathit{x}|\mathit{\xi }+\mathrm{d}\mathit{\xi })$ is defined as [21]

$$\mathrm{d}{s}^2=\sum _{i,j=1}^n{g}_{ij}(\mathit{\xi })\mathrm{d}{\xi }^i\mathrm{d}{\xi }^j.$$

For a smooth manifold, the notion of affine connections permits a covariant differential calculus on its tangent bundle. A statistical manifold S possesses a one-parameter family of affine connections related to the Fisher metric, named the $\alpha$-connections (see [9,22]). For an arbitrary real number $\alpha$, the $\alpha$-connection ${\mathsf{\nabla }}^{(\alpha )}$ can be given by the Christoffel symbols of the first kind

$${\Gamma }_{ij,k}^{(\alpha )}(\mathit{\theta })={\mathbb{E}}_{\mathit{\xi }}\left[\left(\frac{{\partial }^{2}logp(\mathit{x}|\mathit{\xi })}{\partial {\xi }^i\partial {\xi }^j}+\frac{1-\alpha }{2}\frac{\partial logp(\mathit{x}|\mathit{\xi })}{\partial {\xi }^i}\frac{\partial logp(\mathit{x}|\mathit{\xi })}{\partial {\xi }^j}\right)\frac{\partial logp(\mathit{x}|\mathit{\xi })}{\partial {\xi }^k}\right],$$

(2)

for $i,j,k=1,\dots ,n$. Specially, the 0-connection ${\mathsf{\nabla }}^{(0)}$ is exactly the Riemannian (or Levi–Civita) connection with respect to the Fisher metric, and the exponential connection ${\mathsf{\nabla }}^{(1)}$ and the mixture connection ${\mathsf{\nabla }}^{(-1)}$ also have good applications in statistical analysis (e.g., [23,24]).

As detailed in [9], the geometric structure given by the Fisher metric and the $\alpha$-connections obeys the following two invariance principles:

(1)

It is invariant under one-to-one reparameterizations.

(2)

It is invariant under reduction to sufficient statistics.

2.2. Geometric Structure of Exponential Family

An exponential family is a statistical model S consisting of probability densities of the form

$$p(\mathit{x}|\mathit{\theta })=exp\left\{C(\mathit{x})+\sum _{i=1}^n{\theta }^i{F}_i(\mathit{x})-\psi (\mathit{\theta })\right\},$$

(3)

where $C(\mathit{x})$ and $F_i(\mathit{x}),i=1,\dots ,n$, are real-valued functions of $\mathit{x}$, $\mathit{\theta }=[{\theta }^1,\dots ,{\theta }^n]$ are the so-called natural parameters, and $\psi (\mathit{\theta })$ corresponds to the normalization constant. Besides, the $n+1$ functions $1,F_1(\mathit{x}),\dots ,F_n(\mathit{x})$ are generally required to be linearly independent [8]. The exponential families include many common statistical models, such as Gaussian, Poisson, Bernoulli, Gamma, and Dirichlet distributions.

For an exponential family S, let us investigate its geometric structure with respect to the Fisher metric g and $\alpha$-connection ${\mathsf{\nabla }}^{(\alpha )}$. From Equation (3), we can obtain

$$\frac{\partial logp(\mathit{x}|\mathit{\theta })}{\partial {\theta }^i}=F_i(\mathit{x})-\frac{\partial \psi (\mathit{\theta })}{\partial {\theta }^i}\text{and}\frac{{\partial }^{2}logp(\mathit{x}|\mathit{\theta })}{\partial {\theta }^i\partial {\theta }^j}=-\frac{{\partial }^2\psi (\mathit{\theta })}{\partial {\theta }^i\partial {\theta }^j}.$$

(4)

Note that in information geometry, the differentiation and integration of $p(\mathit{x}|\mathit{\theta })$ are customarily assumed to be interchangeable [9]. We can easily verify that an exponential family satisfies this assumption by reference to [25]. Thus from Equations (1) and (4), it is possible to write the components of g as

$$g_{ij}(\mathit{\theta })=-\mathbb{E}\left[\frac{{\partial }^{2}logp(\mathit{x}|\mathit{\theta })}{\partial {\theta }^i\partial {\theta }^j}\right]=\frac{{\partial }^2\psi (\mathit{\theta })}{\partial {\theta }^i\partial {\theta }^j}.$$

(5)

Further computations by Equation (2) give the expressions for ${\mathsf{\nabla }}^{(\alpha )}$. Especially, we have ${\Gamma }_{ij,k}^{(1)}=0$ for $i,j,k=1,\dots ,n$.

In essence, the differential geometry with respect to the Fisher metric and the 0-connection coincides with the usual Riemannian geometry. For all exponential families, it is difficult to derive a general theory of their Riemannian geometric structures. Individually, however, the properties of univariate and multivariate Gaussian distributions are well-studied under the framework of Riemannian geometry in [26].

Amari and Nagaoka [9,19] propose the analysis of the geometric structures of statistical models through duality. Typically, an exponential family S has a dually flat structure $(g,{\mathsf{\nabla }}^{(\pm 1)})$. That is to say, S is flat with respect to the dual connections ${\mathsf{\nabla }}^{(1)}$ and ${\mathsf{\nabla }}^{(-1)}$. On the dual flat space $(S,g,{\mathsf{\nabla }}^{(\pm 1)})$, the natural parameters $\mathit{\theta }$ constitute the 1-affine coordinate system, while the $(-1)$-affine coordinate system can be given by the expectation parameters $\mathit{\eta }=[{\eta }_1,\dots ,{\eta }_n]=\mathbb{E}[\mathit{x}]$. Furthermore, we can prove that the two potential functions provided by $\psi (\mathit{\theta })$ and $\varphi (\mathit{\eta })=\mathit{\theta }·\mathit{\eta }-\psi (\mathit{\theta })$ satisfy

$$\begin{array}{c}{\eta }_i=\frac{\partial \psi (\mathit{\theta })}{\partial {\theta }^i},{\theta }^i=\frac{\partial \varphi (\mathit{\eta })}{\partial {\eta }_i}\hfill \\ g_{ij}=\frac{{\partial }^2\psi (\mathit{\theta })}{\partial {\theta }^i\partial {\theta }^j},g^{ij}=\frac{{\partial }^2\varphi (\mathit{\eta })}{\partial {\eta }_i\partial {\eta }_j},\hfill \end{array}$$

(6)

$$\begin{array}{c}D(p,q)=\psi ({\mathit{\theta }}_q)+\varphi ({\mathit{\eta }}_p)-{\mathit{\theta }}_q·{\mathit{\eta }}_p,\hfill \end{array}$$

(7)

where $g^{ij}$ is the $(i,j)$-th entry of the inverse Fisher information matrix ${\mathit{G}}^{-1}$, ${\mathit{\theta }}_q$ are the 1-affine coordinates of $q\in S$, ${\mathit{\eta }}_p$ are the $(-1)$-affine coordinates of $p\in S$, and $D(p,q)$ denotes the KL divergence from p to q.

2.3. $\alpha$-Geodesics

In the presence of an affine connection, geodesics are defined to be curves whose tangent vectors remain parallel if they are transported along it. Specifically, an $\alpha$-geodesic $\mathit{\gamma }(t)$ is characterized by the equation

$$\frac{{\mathsf{\nabla }}^{(\alpha )}\dot{\mathit{\gamma }}}{\mathrm{d}t}(t)=0,$$

(8)

where $\dot{\mathit{\gamma }}$ denotes the derivative with respect to t.

It is well-known in differential geometry that the (locally) shortest curve between two points of the manifold (if it exists) is a 0-geodesic (namely the geodesic in usual Riemannian geometry; e.g., [27]). Usually, a description is used that the 0-geodesic has “constant speed”, since its velocity fields have constant Riemannian norm. Under certain boundary conditions, when $\alpha =0$, an explicit solution to Equation (8) for univariate or multivariate Gaussian distributions has been derived in [26,28,29]. However, it seems complicated to give out a general closed form of the 0-geodesics for all exponential families.

From the viewpoint of flatness, the manifold S of exponential family is dually flat with respect to ${\mathsf{\nabla }}^{(\pm 1)}$. We can know from [19] that any $(-1)$- or 1-geodesic on S is a straight line with respect to the corresponding affine coordinate system. Thus, in terms of the $\mathit{\theta }$ and $\mathit{\eta }$ coordinates, the $\pm 1$-geodesics ${\mathit{\gamma }}_{p,q}^{(1)}$ and ${\mathit{\gamma }}_{p,q}^{(-1)}$ connecting any $p,q\in S$ can be expressed by

$$\begin{array}{c}\hfill {\mathit{\gamma }}_{p,q}^{(1)}:\mathit{\theta }(t)=(1-t){\mathit{\theta }}_p+t{\mathit{\theta }}_q,\\ \hfill {\mathit{\gamma }}_{p,q}^{(-1)}:\mathit{\eta }(t)=(1-t){\mathit{\eta }}_p+t{\mathit{\eta }}_q,\end{array}$$

(9)

with the parameter $t\in [0,1]$. Note that in Equation (9), the velocity fields $\dot{\mathit{\theta }},\dot{\mathit{\eta }}$ are constant along t, but ${\mathit{\gamma }}_{p,q}^{(1)}$ and ${\mathit{\gamma }}_{p,q}^{(-1)}$ do not have constant speeds.

Example 1 (Univariate Gaussian Distribution).

The probability density of a univariate Gaussian distribution with mean μ and variance ${\sigma }^2$ can be expressed as

$$\begin{array}{c}\hfill p(x|\mu ,{\sigma }^2)=\frac{1}{2\pi \sigma }exp\left\{-\frac{{(x-\mu )}^2}{2{\sigma }^2}\right\}=exp\left\{\sum _{i=1}^2{\theta }^i{F}_i(x)-\psi (\mathit{\theta })\right\},\end{array}$$

where $\mathit{\theta }=[{\theta }^1,{\theta }^2]=[\frac{\mu }{{\sigma }^2},-\frac{1}{2{\sigma }^2}],F_1(x)=x,F_2(x)=x^2$, and

$$\psi (\mathit{\theta })=-\frac{{({\theta }^1)}^2}{4{\theta }^2}-\frac{1}{2}log(-{\theta }^2)+\frac{1}{2}log\pi .$$

Further, we can compute the expectation parameters $\mathit{\eta }=\left[\mathbb{E}[x],\mathbb{E}[x^2]\right]=[\mu ,{\mu }^2+{\sigma }^2]$. Figure 1 illustrates the 0- and $\pm 1$-geodesics on the manifold of univariate Gaussian distributions.

2.4. The Length and Energy of a Curve

The length of a piecewise smooth curve $\mathit{\gamma }(t)=[{\xi }^1(t),\dots ,{\xi }^n(t)],t\in [a,b]$ is defined as

$$L(\mathit{\gamma })={\int }_a^b\sqrt{\sum _{i,j}{g}_{ij}{\dot{\xi }}^i{\dot{\xi }}^j}\mathrm{d}t.$$

(10)

In some occasions, it may be more convenient to work with another related quantity termed the “energy function”, which is given by the integration of the Fisher information along the curve:

$$E(\mathit{\gamma })={\int }_a^b\sum _{i,j}{g}_{ij}{\dot{\xi }}^i{\dot{\xi }}^j\mathrm{d}t.$$

(11)

Minimizing energy turns out to be equivalent to minimizing length, and both lead to the 0-geodesics on the manifold.

Remark 2.

On a manifold, both the length and energy of a curve $\mathit{\gamma }(t)$ do not change under different coordinate systems or model parameterizations, but the latter will depend on the curve parametrization $t\mapsto \mathit{\gamma }(t)$ [30].

Now we discuss the lengths and energies of the 0- and $\pm 1$-geodesics on the manifold S of exponential family. If there exists a unique 0-geodesic connecting $p,q\in S$, the length of this 0-geodesic segment equals the well-known Rao distance between p and q [26], written as $\rho (p,q)$. From Remark 2, the energy of a curve is meaningless without specifying its parametrization. What we mainly consider in this paper are the 0- and $\pm 1$ geodesics parameterized by $[0,1]$. Since the 0-geodesics have constant speed, it means that the integrands in Equations (10) and (11) are constant over t. Thus, given a 0-geodesic ${\mathit{\gamma }}_{p,q}^{(0)}:[0,1]\to S$ connecting p and q, its length and energy have the following relationship [30]:

$$E({\mathit{\gamma }}_{p,q}^{(0)})=L^2({\mathit{\gamma }}_{p,q}^{(0)}).$$

The $\pm 1$-geodesics on S have explicit expressions, shown in Equation (9). Furthermore, the theorem given below describes their energy functions.

Theorem 3.

The energies of the $\pm 1$-geodesics ${\mathit{\gamma }}_{p,q}^{(\pm 1)}$ given in Equation (9) are equal to the Jeffreys divergence $J(p,q)$ between p and q.

Proof. 

By using Equation (5), the energy of ${\mathit{\gamma }}_{p,q}^{(1)}$ connecting $p=\mathit{\theta }(0)$ and $q=\mathit{\theta }(1)$ is

$$\begin{array}{cc}\hfill E({\mathit{\gamma }}_{p,q}^{(1)})& ={\int }_0^1\sum _{i,j=1}^n{g}_{ij}{\dot{\theta }}^i{\dot{\theta }}^j\mathrm{d}t={\int }_0^1\sum _{i,j=1}^n\frac{{\partial }^2\psi (\mathit{\theta })}{\partial {\theta }^i\partial {\theta }^j}\frac{\mathrm{d}{\theta }^i}{\mathrm{d}t}\frac{\mathrm{d}{\theta }^j}{\mathrm{d}t}\mathrm{d}t\hfill \\ & ={\int }_0^1\ddot{\psi }(\mathit{\theta }(t))\mathrm{d}t=\dot{\psi }(\mathit{\theta }(t)){|}_{t=0}^{t=1}\hfill \\ & =\sum _{i=1}^n\frac{\partial \psi (\mathit{\theta }(t))}{\partial {\theta }^i}\frac{\mathrm{d}{\theta }^i}{\mathrm{d}t}{|}_{t=0}^{t=1}\hfill \\ & =({\mathit{\eta }}_q-{\mathit{\eta }}_p)·({\mathit{\theta }}_q-{\mathit{\theta }}_p),\hfill \end{array}$$

where the last equality above follows from Equations (6) and (9). On the other side, according to the expression of $D(p,q)$ in Equation (7), we have

$$\begin{array}{cc}\hfill J(p,q)& \triangleq D(p,q)+D(q,p)\hfill \\ & =(\psi ({\mathit{\theta }}_q)+\varphi ({\mathit{\eta }}_q))+(\psi {(\mathit{\theta })}_p+\varphi ({\mathit{\eta }}_p))-{\mathit{\theta }}_q·{\mathit{\eta }}_p-{\mathit{\theta }}_p·{\mathit{\eta }}_q\hfill \\ & ={\mathit{\theta }}_q·{\mathit{\eta }}_q+{\mathit{\theta }}_p·{\mathit{\eta }}_p-{\mathit{\theta }}_q·{\mathit{\eta }}_p-{\mathit{\theta }}_p·{\mathit{\eta }}_q\hfill \\ & =({\mathit{\eta }}_q-{\mathit{\eta }}_p)·({\mathit{\theta }}_q-{\mathit{\theta }}_p).\hfill \end{array}$$

Then, $E({\mathit{\gamma }}_{p,q}^{(1)})=J(p,q)$. Similarly, we can prove the other half of this theorem. ☐

As an example, on the manifold of univariate Gaussian distributions, Figure 2 shows the energy differences along the $\alpha$-geodesics parameterized by $[0,1]$, for $\alpha =-1,0,1$. We have known that the 0-geodesic path follows the minimum energy variation, which is also demonstrated in Figure 2. Below is a straightforward corollary of Theorem 3.

Corollary 4.

If p and q belong to the same exponential family, then $J(p,q)\ge {\rho }^2(p,q)$.

3. Intrinsic Bayesian Analysis

3.1. Two Intrinsic Losses

Suppose that the observed data $\mathit{x}$ are generated by $p(\mathit{x}|\mathit{\xi })$, for some $\mathit{\xi }\in \mathbf{\Xi }$. Let ${\mathit{\xi }}_a$ be the actual (unknown) value of the parameter, and ${\mathit{\xi }}_g\in \mathbf{\Xi }$ be a given value which may represent an estimator or a hypothesis. For the problems of statistical inference, we define a loss function $l({\mathit{\xi }}_a,{\mathit{\xi }}_g)$, aiming at measuring the consequences of estimating ${\mathit{\xi }}_a$ by ${\mathit{\xi }}_g$ in statistical estimation [5] or judging the compatibility of ${\mathit{\xi }}_g$ with the observations $\mathit{x}$ in hypothesis testing [3].

Many conventional loss functions (such as the squared error loss, the zero-one loss) are defined to compare ${\mathit{\xi }}_a$ and ${\mathit{\xi }}_g$ in the parameter space, while an intrinsic loss function tends to directly measure the dissimilarity between $p(\mathit{x}|{\mathit{\xi }}_a)$ and $p(\mathit{x}|{\mathit{\xi }}_g)$. As customary, the intrinsic losses are required to be invariant under one-to-one transformations of either $\mathit{x}$ or $\mathit{\xi }$ (e.g. [2,5]). Let $S=\{p(\mathit{x}|\mathit{\xi }):\mathit{\xi }\in \mathbf{\Xi }\}$ form an n-dimensional manifold. From the viewpoint of information geometry, the aforementioned criteria are satisfied if a loss function has a well-defined geometrical meaning. Therefore, based on the contents of Section 2.4, we will consider the following two intrinsic losses:

(i)

Intrinsic loss based on the squared Rao distance (hereafter referred to as the Rao loss):

$$l_R({\mathit{\xi }}_a,{\mathit{\xi }}_g)={\rho }^2({\mathit{\xi }}_a,{\mathit{\xi }}_g),$$

(12)

which stands for the energy difference along a 0-geodesic connecting ${\mathit{\xi }}_a$ and ${\mathit{\xi }}_g$ on S and parameterized by $[0,1]$.

(ii)

Intrinsic loss based on the Jeffreys divergence (hereafter referred to as the Jeffreys loss):

$$l_J({\mathit{\xi }}_a,{\mathit{\xi }}_g)=J({\mathit{\xi }}_a,{\mathit{\xi }}_g),$$

(13)

which stands for the energy difference along a $(-1)$- or 1-geodesic connecting ${\mathit{\xi }}_a$ and ${\mathit{\xi }}_g$ on S and parameterized by $[0,1]$ if S is an exponential family.

Since the Jeffreys divergence is a symmetrized KL divergence, it has all the properties of a metric as defined in topology except the triangle inequality property, and is thus not termed a distance. In recent years, this quantity has sometimes been used for model selection [31,32]. There exists another symmetrized version of KL divergence proposed in [2,5] for intrinsic analysis, which takes a minimum over $\left\{D({\mathit{\xi }}_a,{\mathit{\xi }}_g),D({\mathit{\xi }}_g,{\mathit{\xi }}_a)\right\}$ and thus copes better with the case when $p(\mathit{x}|{\mathit{\xi }}_g)$ and $p(\mathit{x}|{\mathit{\xi }}_g)$ have nested supports. Even so, we prefer to adopt the Jeffreys loss $l_J$ in this paper since it has a better-understood geometrical interpretation.

3.2. Priors

Apart from the loss function, an appropriate choice of the prior distribution plays an equally strong role for Bayesian analysis. A particular type of priors have been developed to cope with noninformative settings, where the term “noninformative” expresses that a prior distribution in the Bayes theory is expected to have minimal effect on the posterior inference [5]. A common noninformative prior for the Bayesian paradigm is the Jeffreys prior, which is proportional to the Riemannian volume corresponding to the Fisher metric [20]. As we have learned from the geometric theory, this is an intrinsic concept and is thus not dependent on the model parametrizations.

In many early works, the Jeffreys priors are not recommended for multiparameter cases. Bernardo [33] has initiated the reference priors in Bayesian inference (also known as objective Bayesian inference), which is widely accepted for various multiparameter Bayesian problems. However, in most multidimensional occasions, the analytical evaluations related to an inference prior appear to be quite cumbersome. Thus, some numerical algorithms are designed to do the computations in objective Bayesian inference (e.g., [34,35]), which seems unfavorable for the theoretical derivations of this paper. Additionally, in Section 4, the applications of Jeffreys priors yield an appropriate scale of the eigenvalues of sample covariance matrix, and also reproduce the classic constant false alarm ratio (CFAR) detector in radar detection theory. Hence, we believe that the Jeffreys prior may give a different consequence in intrinsic Bayesian analysis.

3.3. Intrinsic Bayesian Analysis

Let $p(\mathit{\xi }|\mathit{x})\propto p(\mathit{x}|\mathit{\xi })p(\mathit{\xi })$ be the posterior distribution with respect to the prior $p(\mathit{\xi })$. From a Bayesian viewpoint, the corresponding posterior expected loss is

$$r({\mathit{\xi }}_g,\mathit{x})={\int }_{\Xi }l(\mathit{\xi },{\mathit{\xi }}_g)p(\mathit{\xi }|\mathit{x})\mathrm{d}\mathit{\xi }.$$

Considering the invariance criteria, we take the loss function as $l_R$ or $l_J$ and use the Jeffreys prior ${\pi }_J$. This paper will discuss two aspects of intrinsic Bayesian inference: point estimation and hypothesis testing.

As formulated in [3], a hypothesis testing problem is to decide how the observed data $\mathit{x}$ are compatible with the null hypothesis. Specifically, when the null hypothesis contains only one value ${\mathit{\xi }}_0$, an intrinsic Bayesian approach to this hypothesis test is based on the positive statistic $r({\mathit{\xi }}_0,\mathit{x})$, which suggests to reject the null hypothesis if $r({\mathit{\xi }}_0,\mathit{x})>\gamma$ with some threshold $\gamma$; for the composed null hypothesis case, it is suggested to take ${min}_{{\mathit{\xi }}_0\in {\mathbf{\Xi }}_0}r({\mathit{\xi }}_0,\mathit{x})$ as the test statistic, where ${\mathbf{\Xi }}_0$ denotes the null parameter space.

When we come to deal with a point estimation problem, the Bayes estimator of $\mathit{\xi }$ is obtained by minimizing the posterior expected loss function:

$$\widehat{\mathit{\xi }}(\mathit{x})=\underset{\mathit{\xi }\in \mathbf{\Xi }}{argmin}r(\mathit{\xi },\mathit{x}),$$

from which the obtained estimator $\widehat{\mathit{\xi }}(\mathit{x})$ is invariant under invertible transformations of either $\mathit{x}$ or $\mathit{\xi }$. Besides, the property of $\widehat{\mathit{\xi }}(\mathit{x})$ can be further analyzed using the intrinsic estimation theory.

Before finishing this subsection, it is convenient to introduce some basic concepts of intrinsic estimation theory. Fix a $\mathit{\xi }\in \Xi$ and let $T_{\mathit{\xi }}S$ be the tangent space at $\mathit{\xi }$. Associated with a considered connection ∇, the corresponding geodesic curve with starting point $\mathit{\xi }$ and initial direction $\mathit{v}\in T_{\mathit{\xi }}S$ is denoted by ${\mathit{\gamma }}_{\mathit{v}}(t),t\in [0,1]$. Such a curve exists when $\mathit{v}$ belongs to a small neighborhood of the origin at $T_{\mathit{\xi }}S$. In this case, the exponential map is defined as ${Exp}_{\mathit{\xi }}(\mathit{v})={\mathit{\gamma }}_{\mathit{v}}(1)$. Given an estimator $\widehat{\mathit{\xi }}(\mathit{x})$, the estimator vector field is induced on S through the inverse of exponential map: ${\mathcal{A}}_{\mathit{\xi }}(\mathit{x})={Exp}_{\mathit{\xi }}^{-1}(\widehat{\mathit{\xi }}(\mathit{x}))$. Further, the bias vector field is defined as

$${\mathcal{B}}_{\mathit{\xi }}={\mathbb{E}}_{\mathit{\xi }}[{\mathcal{A}}_{\mathit{\xi }}].$$

If $\parallel {\mathcal{B}}_{\mathit{\xi }}\parallel =0$ for any $\mathit{\xi }\in \mathbf{\Xi }$, the estimator $\widehat{\mathit{\xi }}(\mathit{x})$ is called intrinsically unbiased. Obviously, these definitions above are dependent on the specific choice of ∇ [18]. When a flat connection is considered, an estimator will be intrinsically unbiased if and only if it is unbiased under the corresponding affine coordinate system. Generally speaking, however, the notion of intrinsic unbiasedness is widely acknowledged only in the Riemannian case (namely, when ∇ is the Riemannian connection). Thus, in this paper, when we say that an estimator is intrinsically unbiased, we mean that this is true with regard to the 0-connection. The intrinsic MSE is defined by the mean square of the Rao distance

$$\mathrm{MSE}(\widehat{\mathit{\xi }}(\mathit{x}))=\mathbb{E}[{\rho }^2(\mathit{\xi },\widehat{\mathit{\xi }}(\mathit{x}))],$$

(14)

where, in the Bayesian sense, $\mathbb{E}$ denotes the expectation taken over both $\mathit{x}$ and $\mathit{\xi }$. An intrinsic version of the CRB gives a lower bound on the intrinsic MSE performance of any intrinsically unbiased or biased estimator. The relevant developments can be found in [17,18,36].

4. Applications

4.1. Covariance Estimation

Let ${\mathit{x}}_1,\dots ,{\mathit{x}}_n$ be random samples of size n from a zero-mean p-variate Gaussian distribution with unknown covariance matrix $\mathbf{\Sigma }$. Consider the Jeffreys prior distribution for $\mathbf{\Sigma }$ ([37], p. 426), ${\pi }_J(\mathbf{\Sigma })={\left|\mathbf{\Sigma }\right|}^{-\frac{p+1}{2}}$. A sufficient statistic is provided by

$$\mathit{S}=\frac{1}{n}\sum _{i=1}^n{\mathit{x}}_i{\mathit{x}}_i^T,$$

which is recognized as the sample covariance matrix of the data set [38]. Assume that $n>p$; then, $\mathit{S}$ is positive definite with probability one. Thus, the corresponding posterior distribution based on the Jeffreys prior is easily found to be

$$\pi (\mathbf{\Sigma }|\mathit{S})\propto p(\mathit{S}|\mathbf{\Sigma }){\pi }_J(\mathbf{\Sigma })\propto {\left|\mathbf{\Sigma }\right|}^{-\frac{n+p+1}{2}}exp\left\{-\frac{n}{2}tr({\mathbf{\Sigma }}^{-1}\mathit{S})\right\},$$

which follows an inverse Wishart distribution with n degrees of freedom and scale matrix $n\mathit{S}$.

Now we restrict to consider a family of covariance estimators having the form ${\mathbf{\Sigma }}_g=a\mathit{S},a\in (0,+\infty )$, which contain the MLE $\mathit{S}$. The Rao distance between ${\mathbf{\Sigma }}_1$ and ${\mathbf{\Sigma }}_2$ is [39]

$$\rho ({\mathbf{\Sigma }}_1,{\mathbf{\Sigma }}_2)=\frac{1}{\sqrt{2}}\sqrt{tr\left({log}^2({\mathbf{\Sigma }}_1^{-\frac{1}{2}}{\mathbf{\Sigma }}_2{\mathbf{\Sigma }}_1^{-\frac{1}{2}})\right)},$$

(15)

while the Jeffreys divergence is [40]

$$J({\mathbf{\Sigma }}_1,{\mathbf{\Sigma }}_2)=\frac{1}{2}\left(tr({\mathbf{\Sigma }}_1^{-1}{\mathbf{\Sigma }}_2)+tr({\mathbf{\Sigma }}_2^{-1}{\mathbf{\Sigma }}_1)-2p\right).$$

Taking the intrinsic losses into account, we can obtain an intrinsic estimator by minimization of the posterior expected Rao loss function:

$${\widehat{\mathbf{\Sigma }}}_R=\underset{{\mathbf{\Sigma }}_g\in \left\{a\mathit{S}\right|a>0\}}{argmin}\int l_R(\mathbf{\Sigma },{\mathbf{\Sigma }}_g)\pi (\mathbf{\Sigma }|\mathit{S})\mathrm{d}\mathbf{\Sigma }=\underset{a\in (0,+\infty )}{argmin}{\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[{\rho }^2(a\mathit{S},\mathbf{\Sigma })\right],$$

(16)

where ${\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}$ signifies the expectation with respect to $\pi (\mathbf{\Sigma }|\mathit{S})$. By Equation (15), we have

$${\rho }^2(a\mathit{S},\mathbf{\Sigma })=\frac{1}{2}tr\left({log}^2(a{\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}})\right)=\frac{1}{2}\sum _{i=1}^p{log}^2(a{\lambda }_i),$$

where ${\lambda }_i,i=1,\dots ,p$ denote the p eigenvalues of the matrix ${\mathbf{\Sigma }}^{-1}\mathit{S}$. In fact, it proves quite difficult to directly solve the conditional expectation in Equation (16) as a closed form. However, if we let $c=loga$, it is easy to show that the objective function in Equation (16) is a strictly convex function with respect to c. Thus, we can resort to the Lagrange equation to seek the minimum point. Write the posterior expected Rao loss function as

$$r_R(c)={\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[{\rho }^2(a\mathit{S},\mathbf{\Sigma })\right]={\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[\frac{1}{2}\sum _{i=1}^p{(c+log{\lambda }_i)}^2\right].$$

Then, let

$$\frac{\mathrm{d}{r}_R}{\mathrm{d}c}=\frac{\mathrm{d}}{\mathrm{d}c}{\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[\frac{1}{2}\sum _{i=1}^p{(c+log{\lambda }_i)}^2\right]=0.$$

By Leibniz’s rule [41], we can interchange the integral and differential operators, obtaining

$$\begin{array}{cc}\hfill \frac{\mathrm{d}{r}_R}{\mathrm{d}c}& ={\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[\frac{1}{2}\frac{\mathrm{d}}{\mathrm{d}c}\sum _{i=1}^p{(c+log{\lambda }_i)}^2\right]={\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[\sum _{i=1}^p(c+log{\lambda }_i))\right]\hfill \\ & =pc+{\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[log|{\mathbf{\Sigma }}^{-1}\mathit{S}|\right]=0.\hfill \end{array}$$

(17)

Since $\mathbf{\Sigma }|\mathit{S}\sim \mathrm{Inverse}\mathrm{Wishart}(n,n\mathit{S})$, ${\mathit{S}}^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{S}}^{\frac{1}{2}}|\mathit{S}\sim \mathrm{Wishart}(n,n^{-1}{\mathit{I}}_p)$. Combining with the fact that $|{\mathit{S}}^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{S}}^{\frac{1}{2}}|=|{\mathbf{\Sigma }}^{-1}\mathit{S}|$, we can calculate the conditional expectation in Equation (17) by the law of log-determinant of a Wishart matrix [42]:

$${\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[log|{\mathbf{\Sigma }}^{-1}\mathit{S}|\right]=\sum _{i=1}^p\psi \left(\frac{n+1-i}{2}\right)+plog2-plogn,$$

where $\psi (·)$ is the well-known digamma function defined as $\psi (x)=\frac{\mathrm{d}}{\mathrm{d}x}log\Gamma (x)$. Thus, from Equation (17), the intrinsic covariance estimator based on the Rao loss is ${\widehat{\mathbf{\Sigma }}}_R={\widehat{a}}_R\mathit{S}$, with

$${\widehat{a}}_R=exp(\widehat{c})=\frac{n}{2}exp\left\{-\frac{1}{p}\sum _{i=1}^p\psi \left(\frac{n+1-i}{2}\right)\right\}.$$

(18)

Alternatively, if we consider the Jeffreys loss function $l_J$, then

$$l_J(a\mathit{S},\mathbf{\Sigma })=\frac{1}{2}\left(atr({\mathbf{\Sigma }}^{-1}\mathit{S})+a^{-1}tr(\mathbf{\Sigma }{\mathit{S}}^{-1})-2p\right).$$

It is easy to calculate that ${\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[tr({\mathbf{\Sigma }}^{-1}\mathit{S})\right]=p$, and when $n>p+1$,

$${\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[tr(\mathbf{\Sigma }{\mathit{S}}^{-1})\right]=\frac{np}{n-p-1}.$$

Therefore, the intrinsic covariance estimator based on the Jeffreys loss is ${\widehat{\mathbf{\Sigma }}}_J={\widehat{a}}_J\mathit{S}$, with

$${\widehat{a}}_J=\sqrt{\frac{n}{n-p-1}}.$$

In

Table 1. The scale factors of sample covariance matrix for some values of n and $p=10$.

n	${\widehat{\mathit{a}}}_{\mathit{R}}$	${\widehat{\mathit{a}}}_{\mathit{J}}$	n	${\widehat{\mathit{a}}}_{\mathit{R}}$	${\widehat{\mathit{a}}}_{\mathit{J}}$
12	$2.06698$	$3.46410$	50	$1.12585$	$1.13228$
13	$1.87853$	$2.54951$	100	$1.05867$	$1.06000$
14	$1.74997$	$2.16025$	200	$1.01114$	$1.02869$
16	$1.58326$	$1.78885$	500	$1.04598$	$1.01118$
18	$1.47864$	$1.60357$	${10}^3$	$1.00553$	$1.00555$
20	$1.40640$	$1.49071$	${10}^4$	$1.00055$	$1.00055$

, the scale factors ${\widehat{a}}_R$ and ${\widehat{a}}_J$ are evaluated for various values of n and $p=10$.

We now come to examine the bias and efficiency of these two intrinsic estimators.

Proposition 5.

Let $\mathit{W}\sim \mathrm{Wishart}(v,{\mathit{I}}_p)$ with $v\ge p$, then $\mathbb{E}[{log}^k(\mathit{W})]=m{\mathit{I}}_p$, where m is a constant decided by $p,k$, and v.

Proof. 

For a $p\times p$ unitary matrix $\mathit{U}$, the random matrix $\mathit{U}\mathit{W}{\mathit{U}}^T$ follows the same distribution with $\mathit{W}$. Thus,

$$\mathbb{E}\left[{log}^k(\mathit{W})\right]=\mathbb{E}\left[{log}^k(\mathit{U}\mathit{W}{\mathit{U}}^T)\right].$$

Recall the important fact that $log({\mathit{A}}^{-1}\mathit{B}\mathit{A})={\mathit{A}}^{-1}log(\mathit{B})\mathit{A}$, where $\mathit{A}$ and $\mathit{B}$ are two nonsingular matrices. Thus, a further step yields

$$\mathbb{E}\left[{log}^k(\mathit{W})\right]=\mathit{U}\mathbb{E}\left[{log}^k(\mathit{W})\right]{\mathit{U}}^T.$$

Note that the equality above holds for an arbitrary unitary matrix $\mathit{U}$; then, we can conclude that $\mathbb{E}\left[{log}^k(\mathit{W})\right]$ must be a scalar multiple of the identity matrix. ☐

Theorem 6.

The covariance estimator ${\widehat{\mathbf{\Sigma }}}_R$ given by Equation (18) is intrinsically unbiased.

Proof. 

For a p-dimensional positive-definite matrix $\mathit{X}$, the inverse of the exponential map defined on the manifold of positive definite matrices is given by [43]

$${exp}_{\mathbf{\Sigma }}^{-1}(\mathit{X})={\mathbf{\Sigma }}^{\frac{1}{2}}log({\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{X}{\mathbf{\Sigma }}^{-\frac{1}{2}}){\mathbf{\Sigma }}^{\frac{1}{2}}.$$

Thus, we have

$$\begin{array}{cc}\hfill \mathbb{E}[{exp}_{\mathbf{\Sigma }}^{-1}({\widehat{\mathbf{\Sigma }}}_R)]& =\mathbb{E}[{\mathbf{\Sigma }}^{\frac{1}{2}}log({\widehat{a}}_R{\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}}){\mathbf{\Sigma }}^{\frac{1}{2}}]\hfill \\ & ={\mathbf{\Sigma }}^{\frac{1}{2}}(log[{\widehat{a}}_R{\mathit{I}}_p]+\mathbb{E}[log({\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}})]){\mathbf{\Sigma }}^{\frac{1}{2}}.\hfill \end{array}$$

(19)

Generally, it is difficult to directly solve the expectation of the logarithm of a Wishart matrix. However, since $n{\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}}\sim \mathrm{Wishart}(n,{\mathit{I}}_p)$, by Proposition 5, the expectation in Equation (19) has this form

$$\mathbb{E}[log(n{\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}})]=m{\mathit{I}}_p$$

(20)

with a certain constant m. In addition, since

$$tr\left(\mathbb{E}\left[log[n{\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}}]\right]\right)=\mathbb{E}\left[log|{\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}}|\right]=\sum _{i=1}^p\psi \left(\frac{n+1-i}{2}\right)+plog2,$$

we can obtain

$$m=\frac{1}{p}\sum _{i=1}^p\psi \left(\frac{n+1-i}{2}\right)+log2.$$

(21)

Substituting Equations (18), (20) and (21) into Equation (19), we have ${\mathbb{E}}_{\mathbf{\Sigma }}\left[{exp}_{\mathbf{\Sigma }}^{-1}({\widehat{\mathbf{\Sigma }}}_R)\right]=\mathbf{0}$, which indicates that the covariance estimator ${\widehat{\mathbf{\Sigma }}}_R$ is intrinsically unbiased. ☐

Given a real number $a>0$, the posterior expected Rao loss of $a\mathit{S}$ can be computed by

$$\begin{array}{cc}\hfill {\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}[{\rho }^2(a\mathit{S},\mathbf{\Sigma })]& ={\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[\frac{1}{2}tr\left({log}^2(a{\mathbf{\Sigma }}^{-\frac{1}{2}}\mathit{S}{\mathbf{\Sigma }}^{-\frac{1}{2}})\right)\right]=\frac{1}{2}tr\left({\mathbb{E}}_{\mathbf{\Sigma }|\mathit{S}}\left[\frac{1}{2}{log}^2(a{\mathit{S}}^{\frac{1}{2}}{\mathbf{\Sigma }}^{-1}{\mathit{S}}^{\frac{1}{2}})\right]\right).\hfill \end{array}$$

Using Proposition 5 again, we know that the right-hand side of the second equality above is constant on the sample space. Thus, the posterior expected Rao losses of ${\widehat{\mathbf{\Sigma }}}_R,{\widehat{\mathbf{\Sigma }}}_J$, and $\mathit{S}$ are equal to their Bayes risks under Rao loss that average the Rao loss functions over both sample space and parameter space. Similar consequences can be deduced for the posterior expected Jeffreys losses of ${\widehat{\mathbf{\Sigma }}}_R,{\widehat{\mathbf{\Sigma }}}_J,\mathit{S}$, and their Bayes risks under Jeffreys loss. It is still difficult to seek an explicit expression for the posterior expected Rao loss. Hence, a Monte Carlo simulation is used with 1000 trials to calculate the Bayes risks under Rao loss. The comparisons are illustrated in Figure 3.

In Figure 4, we compare the intrinsic MSEs (see Equation (14)) of the considered estimators with an intrinsic version of the CRB given in ([18], Theorem 4), which serves as a lower bound for any intrinsically unbiased covariance estimator. In fact, for our case, the intrinsic MSEs of ${\widehat{\mathbf{\Sigma }}}_R,{\widehat{\mathbf{\Sigma }}}_J$ and $\mathit{S}$ are equal to their Bayes risks under Rao loss. Recalling that the MLE $\mathit{S}$ is not intrinsically unbiased but always asymptotically efficient [18], we can conclude that the Bayes estimator ${\widehat{\mathbf{\Sigma }}}_R$ under Rao loss is intrinsically unbiased and also asymptotically efficient. This property is also demonstrated in Figure 4.

4.2. Range-Spread Target Detection

The target detection in the background of Gaussian noise is a basic problem in radar signal processing. Suppose that an unknown target is spatially distributed across H range cells. Data collected from these H cells—called the primary data—consist of the possible target echoes plus Gaussian noises. As in [44], we assume that a set of secondary data coming from K cells arranged around the target are available, which serve as training samples for noise covariance estimation. Here the homogeneous noise environment is considered, which means that the secondary data are free of signal components and share the same distribution as the noise-only part of the primary data. Therefore, the considered detection problem can be formulated as [44]

$$\begin{array}{c}{H}_0:{\mathit{z}}_i={\mathit{w}}_i,i=1,\dots ,H+K,\hfill \\ H_1:\left\{\begin{array}{cc}{\mathit{z}}_i={\mathit{p}}_i+{\mathit{w}}_i,\hfill & i=1,\dots ,H,\hfill \\ {\mathit{z}}_i={\mathit{w}}_i,\hfill & i=H+1,\dots ,H+K,\hfill \end{array}\right.\hfill \end{array}$$

(22)

where ${\mathit{p}}_i,i=1,\cdots ,H$ denote the N-dimensional unknown signal vectors, and ${\mathit{w}}_i,i=1,\cdots ,H+K$, are independent and identically distributed Gaussian noises with mean zero and unknown covariance matrix $\mathit{M}$. For the brevity of notation, let us introduce the matrix forms

$${\mathit{Z}}_p=[{\mathit{z}}_1,\dots ,{\mathit{z}}_H],{\mathit{Z}}_s=[{\mathit{z}}_{H+1},\dots ,{\mathit{z}}_{H+K}],\mathit{P}=[{\mathit{p}}_1,\dots ,{\mathit{p}}_H].$$

Denote $\mathit{Z}=[{\mathit{Z}}_p,{\mathit{Z}}_s]$ and $\tilde{\mathit{P}}=[\mathit{P},\mathbf{0}]$, then the matrix $\mathit{Z}$ of measurements is Gaussian distributed with mean $\tilde{\mathit{P}}$ and covariance ${\mathit{I}}_{H+K}\otimes \mathit{M}$. In the absence of target, we have $\mathit{P}=\mathbf{0}$ under the null hypothesis. Therefore, the Rao distance between the assumed model and the null model is [45]

$$\rho \left([\tilde{\mathit{P}},{\mathit{I}}_{H+K}\otimes \mathit{M}],[\mathbf{0},{\mathit{I}}_{H+K}\otimes \mathit{M}]\right)=\sqrt{\sum _{i=1}^H{\mathit{p}}_i^T{\mathit{M}}^{-1}{\mathit{p}}_i}.$$

(23)

We shall adopt again the Jeffreys prior distribution for parameters $(\mathit{P},\mathit{M})$ [35]:

$${\pi }_J(\mathit{P},\mathit{M})\propto {\left|\mathit{M}\right|}^{-\frac{N+H+1}{2}}.$$

The joint density for $\mathit{Z}$ is proportional to

$${\left|\mathit{M}\right|}^{-\frac{H+K}{2}}exp\left\{-\frac{1}{2}tr\left({\mathit{M}}^{-1}({\mathit{S}}_1+{\mathit{S}}_2)\right)\right\},$$

where ${\mathit{S}}_1=({\mathit{Z}}_p-\mathit{P}){({\mathit{Z}}_p-\mathit{P})}^T,{\mathit{S}}_2={\mathit{Z}}_s{\mathit{Z}}_s^T$. Thus, the corresponding posterior distribution satisfies

$$\begin{array}{cc}\hfill \pi (\mathit{P},\mathit{M}|\mathit{Z})& \propto {\pi }_J(\mathit{P},\mathit{M})p(\mathit{Z}|\mathit{P},\mathit{M})\propto {\left|\mathit{M}\right|}^{-\frac{N+H+1}{2}}·{\left|\mathit{M}\right|}^{-\frac{H+K}{2}}exp\left\{-\frac{1}{2}tr\left({\mathit{M}}^{-1}({\mathit{S}}_1+{\mathit{S}}_2)\right)\right\}\hfill \\ & ={\left|\mathit{M}\right|}^{-\frac{H}{2}}exp\left\{-\frac{1}{2}tr({\mathit{M}}^{-1}{\mathit{S}}_1)\right\}·{\left|\mathit{M}\right|}^{-\frac{H+K+N+1}{2}}exp\left\{-\frac{1}{2}tr({\mathit{M}}^{-1}{\mathit{S}}_2)\right\}.\hfill \end{array}$$

Since $\pi (\mathit{P},\mathit{M}|\mathit{Z})=\pi (\mathit{P}|\mathit{M},\mathit{Z})\pi (\mathit{M}|\mathit{Z})$, we can conclude that

$$\pi (\mathit{P}|\mathit{M},\mathit{Z})=\text{Normal}({\mathit{Z}}_p,{\mathit{I}}_H\otimes \mathit{M}),\pi (\mathit{M}|\mathit{Z})=\text{Inverse}\text{Wishart}(H+K,{\mathit{S}}_2).$$

(24)

Therefore, the posterior expected Rao loss of determining the null hypothesis is derived as follows:

$$\begin{array}{c}{\mathbb{E}}_{\mathit{P},\mathit{M}|\mathit{Z}}\left[l_R\left([\tilde{\mathit{P}},{\mathit{I}}_{H+K}\otimes \mathit{M}],[\mathbf{0},{\mathit{I}}_{H+K}\otimes \mathit{M}]\right)\right]\hfill \\ ={\mathbb{E}}_{\mathit{M}|\mathit{Z}}\left[{\mathbb{E}}_{\mathit{P}|\mathit{M},\mathit{Z}}\left[\sum _{i=1}^H{\mathit{p}}_i^T{\mathit{M}}^{-1}{\mathit{p}}_i\right]\right]={\mathbb{E}}_{\mathit{M}|\mathit{Z}}\left[\sum _{i=1}^H{\mathit{z}}_i^T{\mathit{M}}^{-1}{\mathit{z}}_i+N\right]\hfill \\ =N+(H+K)\sum _{i=1}^H{\mathit{z}}_i^T{\mathit{S}}_2^{-1}{\mathit{z}}_i.\hfill \end{array}$$

Hence, the decision rule for the compatibility of the null hypothesis with the observed data $\mathit{Z}$ can be given as

$$\text{Reject}{H}_0\text{if}\text{and}\text{only}\text{if}\sum _{i=1}^H{\mathit{z}}_i^T{\mathit{S}}_2^{-1}{\mathit{z}}_i>\gamma ,$$

where the threshold $\gamma$ is chosen by the required false alarm probability. In fact, this result reproduces the two-step generalized likelihood ratio test [46] or Wald test [47], which is verified to have CFAR property [46].

Besides, since in this case,

$$l_J\left([\tilde{\mathit{P}},{\mathit{I}}_{H+K}\otimes \mathit{M}],[\mathbf{0},{\mathit{I}}_{H+K}\otimes \mathit{M}]\right)=l_R\left([\tilde{\mathit{P}},{\mathit{I}}_{H+K}\otimes \mathit{M}],[\mathbf{0},{\mathit{I}}_{H+K}\otimes \mathit{M}]\right),$$

we can derive an equivalent decision rule using the Jeffreys loss.

Finally, as a conclusion of this section, we will conduct a discussion about the relationship between the geometric invariance principle and the CFAR property. If we transform the random variables $\mathit{x}$ to $\mathit{y}$ without altering the structures of statistical model and testing problem, it is equivalent to derive a transformation on the parameter space $\mathbf{\Xi }$. Let a decision statistic be an intrinsic concept in the geometrical meaning. Thus, this statistic is invariant by changing $\mathit{x}$ to $\mathit{y}$, which indicates that it maintains a CFAR behavior over an orbit of the parameter space.

5. Conclusions

In this paper, the Rao loss and Jeffreys loss—which are, respectively, based on the squared Rao distance and the Jeffreys divergence—are uniformly considered in the framework of information geometry. We have elucidated their geometrical meaning based on some results of Riemannian geometry and dualistic geometry. In particular, on the manifold of an exponential family, the Rao loss and Jeffreys loss are essentially the energy differences along $\alpha$-geodesic paths parameterized by $[0,1]$, for $\alpha =-1,0,1$. Based on this, they certainly enjoy the invariance properties under the one-to-one transformations of random variables and model parameters. Subsequently, these two intrinsic loss functions are unitized to develop intrinsic Bayesian analysis of covariance estimation and range-spread target detection. We use the intrinsic losses to derive the scale factor of sample covariance matrix, which leads to an intrinsically unbiased and asymptomatically efficient covariance estimator. On the other hand, the detectors provided by the posterior expected intrinsic losses have been proven to coincide with the classic CFAR detector in radar detection theory. In this respect, it seems to be a novel but promising approach to derive a detector having CFAR property though intrinsic analysis.