Next Article in Journal
Generalized Maximum Entropy Analysis of the Linear Simultaneous Equations Model
Previous Article in Journal
Autonomous Search for a Diffusive Source in an Unknown Structured Environment
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Note on the W-S Lower Bound of the MEE Estimation

1
Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an 710049, China
2
Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA
*
Author to whom correspondence should be addressed.
Entropy 2014, 16(2), 814-824; https://doi.org/10.3390/e16020814
Submission received: 5 November 2013 / Revised: 7 January 2014 / Accepted: 26 January 2014 / Published: 10 February 2014

Abstract

: The minimum error entropy (MEE) estimation is concerned with the estimation of a certain random variable (unknown variable) based on another random variable (observation), so that the entropy of the estimation error is minimized. This estimation method may outperform the well-known minimum mean square error (MMSE) estimation especially for non-Gaussian situations. There is an important performance bound on the MEE estimation, namely the W-S lower bound, which is computed as the conditional entropy of the unknown variable given observation. Though it has been known in the literature for a considerable time, up to now there is little study on this performance bound. In this paper, we reexamine the W-S lower bound. Some basic properties of the W-S lower bound are presented, and the characterization of Gaussian distribution using the W-S lower bound is investigated.
MSC Codes::
62B10

1. Introduction

Let X ∈ ℝn and Y ∈ ℝm be two random vectors, with joint probability density function (PDF) pXY(x, y), where X represents a unknown variable and Y stands for the observation. An optimal estimator of X based on the observation is a function of Y that minimizes a certain cost function. Under the well-known minimum mean square error (MMSE) criterion, the optimal estimator is:

g * = arg min g G E [ E 2 ] = arg min g G E [ ( X g ( Y ) ) 2 ] = arg min g G m n ( x g ( y ) ) 2 p X Y ( x , y ) d x d y
where E[.] denotes the expectation operator, E = Xg(Y) denotes the estimation error, and G denotes the collection of all measurable functions of Y. The MMSE criterion is prevalent in estimation theory due to its mathematical tractability. Under Gaussian assumption, the MMSE criterion yields a linear optimal estimator, which requires only a simple matrix-vector operation [1]. When the data are non-Gaussian, however, the MMSE estimator will be suboptimal and even unacceptable, since it considers only up to second-order statistics for their design.

In order to take into account higher-order statistics in design of estimators, researchers proposed many non-MMSE criteria. The minimum error entropy (MEE) is one of such criteria [29]. Under MEE criterion, the optimal estimator is obtained by minimizing the error entropy, that is:

g = arg min g G H ( E ) = arg min g G n p E ( e ) log p E ( e ) d e = arg min g G E [ log p E ( E ) ]
where H(E) denotes the Shannon entropy [10] of the error E, and pE (.) denotes the error’s PDF, which is:
p E ( e ) = m p X | Y ( e + g ( y ) | y ) p Y ( y ) d y
where pX|Y(. |y) denotes the conditional PDF of X given Y = y, and pY (y) is the marginal PDF of Y. The MEE criterion is invariant with respect to the error’s mean. In practice, the MEE estimator is usually restricted to an unbiased one with zero-mean error. The entropy is a measure of concentration of a distribution, minimizing the error entropy forces the error to gather.

The early work in MEE estimation can be traced back to the late 1960s when Weidemann and Stear [2] studied the use of error entropy as a cost function for analyzing the performance of a general sampled-data estimating systems. Minamide [5] extended Weidemann and Stear’s results to a continuous-time estimating system. Tomita, Kalata, Minamide et al. applied the MEE estimation to linear Gaussian systems, and studied filtering (state estimation), smoothing, and predicting problems from the information theory viewpoint [35]. Some important properties of the MEE estimation were also reported in [1116]. In recent years, MEE has become a popular optimization criterion in the areas of signal processing and machine learning [8,9,1722]. Combining kernel density estimation (KDE) and Renyi’s quadratic entropy yields a computationally simple, nonparametric entropy estimator that has been successfully used in information theoretic learning (ITL) [8].

There is a performance bound on the MEE estimation, which was originally derived by Weidemann and Stear [2], and later was rederived and named the W-S lower bound by Janzura et al. [6]. The W-S lower bound provides a lower bound on the error entropy, although it is not necessarily attained by the MEE estimator for a given joint distribution pXY. This performance bound is nothing but the conditional entropy of the unknown variable X given the observation Y, that is, we have:

H ( E ) H ( X | Y )

The above inequality can be easily derived using Jensen’s inequality. Let Φ (x) = − x log x. We have:

H ( E ) = n Φ ( p E ( x ) ) d x = n Φ ( m p X | Y ( x + g ( y ) | y ) p Y ( y ) d y ) d x ( a ) n [ m Φ ( p X | Y ( x + g ( y ) | y ) ) p Y ( y ) d y ] d x = m [ n Φ ( p X | Y ( x + g ( y ) | y ) ) d x ] p Y ( y ) d y = m [ n Φ ( p X | Y ( x | y ) ) d x ] p Y ( y ) d y = m H ( X | Y = y ) p Y ( y ) d y = H ( X | Y )
where (a) comes from the concavity of Φ(x) and Jensen’s inequality, and H(X | Y = y) denotes the conditional entropy of X given Y = y.

The performance bounds are very important in estimation theory. So far there is, however, little study on the W-S lower bound of the MEE estimation. In this paper, we will present some important properties of the W-S lower bound, and show that this performance bound can be applied to characterize the Gaussian distribution. The rest of the paper is organized as follows: in Section 2, some basic properties of the W-S lower bound are presented. In Section 3, the characterization of the Gaussian distribution using W-S lower bound is investigated. Finally, the conclusions are given in Section 4.

2. Some Properties of the W-S Lower Bound

In the following, we present some properties of the W-S lower bound. First, we present several sufficient and necessary conditions under which the W-S lower bound can be achieved.

Theorem 1: Let X and Y be two random vectors, X ∈ ℝn, Y ∈ ℝm. The MEE estimator = g(Y) of X based on Y achieves the W-S lower bound H (X | Y) if and only if any one of the following properties holds:

(1)

the error E = Xg(Y) is independent of Y;

(2)

X = g(Y) + Z, where Z ∈ ℝn is a random vector that is independent of Y;

(3)

pXY(x, y) =, pZ (xg(y)) pY (y), where pz(.) is a density function that is independent of Y;

(4)

pE(x) =pX|Y(x + g(y)|y).

Proof: (1) Denote I (E;Y) the mutual information between E and Y. It is easy to derive:

I ( E ; Y ) = H ( E ) H ( E | Y ) = H ( E ) H ( X g ( Y ) | Y ) = H ( E ) H ( X | Y )

Hence H(E) =H(X | Y) ⇔ I (E;Y) = 0 The mutual information I (E; Y) equals zero if and only if E and Y are independent, so we conclude that the MEE estimator achieves the W-S lower bound if and only if the error is independent of Y.

(2) As E = Xg (Y), we have X = g(Y) + E. The error entropy H(E) achieves the W-S lower bound H (X | Y) if and only if E is independent of Y (i.e., I(E; Y) = 0), so we have X = g (Y) + Z, where Z ∈ ℝn is independent of Y (let Z = E).

(3) The error entropy achieves the W-S lower bound if and only if X = g(Y) + Z, where Z ∈ ℝn is independent of Y. Denote the density function of Z by pZ (.). The conditional density function of X given Y = y will be pX|Y (x|y) = pZ (xg(y)). Thus pXY (x, y) = pX|Y (x|y) = pz (xg(y)) pY (y).

(4) Since the conditional density function of X given Y is pX|Y (x| y) = pz (xg(y)) = pE (xg(y)), we have pE(x) = pX|Y(x + g(y)| y).

Remark: The properties (2)(4) of Theorem 1 suggest that if the error entropy achieves the W-S lower bound, only the location (or mean) of the conditional density of X given Y = y will depend on y through function g(y), while the shape of the conditional density is always the same as the shape of the error density, which is independent of y.

Theorem 2: Let X and Y be two random vectors, X = [X1, X2,· · ·Xn] ∈ ℝn, Ym. If there exists an MEE estimator of X based on Y that achieves the W-S lower bound H(X|Y), then ∀r = [r1, r2,…, rn] ∈ ℕn, the r -th conditional central moment of X given Y = y is constant over ℝm E = [ ( X 1 μ 1 ) r 1 ( X 2 μ 2 ) r 2 ( X n μ n ) r n | Y = y ] that is, does not depend on y, where μi denotes the conditional mean value of Xi given Y = y.

Proof: If the error entropy achieves the W-S lower bound, the shape of the conditional density of X given Y = y will not depend on y. Thus, the theorem holds since the shape of a density function determines its central moments (Note that the central moments depend only on the shape of a density, and are independent of the location of distribution).

Theorem 3: Let X and Y be two random vectors, X ∈ ℝn, Y ∈ ℝm. If there exists an MEE estimator of X based on Y that achieves the W-S lower bound H(X|Y), then it will be:

X ^ = g ( Y ) = E [ X | Y ] + c
where c ∈ ℝn is a n -dimensional constant vector.

Proof: According to the property (2) of Theorem 1, we have X = g(Y) + Z, where Z ∈ ℝnis a random vector that is independent of Y. It follows easily that:

E [ X | Y ] = E [ g ( Y ) + Z | Y ] = g ( Y ) + E [ Z ]
And hence, g(Y) = E[X|Y] + c, where c = − E[Z].

It has been shown in [15] that the MEE estimator may be non-unique even if the error distribution is restricted to zero-mean (unbiased). However, the following corollary holds.

Corollary 1: Let X and Y be two random vectors, X ∈ ℝn, Y ∈ ℝm. If there exists an MEE estimator of X based on Y that achieves the W-S lower bound H (X|Y), then the unbiased MEE estimator will be unique, and be identical to the MMSE estimator.

Proof: If error E is restricted to zero-mean (i.e., E[E]=0), by Theorem 3, we have g (Y) = E[X|Y] (i.e., c = 0). In this case, the MEE estimator becomes the conditional mean of X given Y (i.e., the MMSE estimator), which is, obviously, unique.

Theorem 4: Let X and Y be two random vectors, X ∈ ℝn, Y ∈ ℝm. If there exists an MEE estimator of X based on Y that achieves the W-S lower bound H (X|Y), then the MEE estimator and the smoothed MEE (SMEE) estimator of X based on Y will be identical.

Proof: According to [16], the SMEE estimator is obtained by minimizing the smoothed MEE criterion H(E + λU), where λ is the smoothing factor, and U is a smoothing variable (see [16] for the detailed description of the smoothing variable) that is independent of X, Y and E. Clearly, the SMEE estimator of X based on Y is identical to the MEE estimator of X + λ U based on Y. Since the MEE estimator of X based on Y achieves the W-S lower bound, we have X = g(Y) + Z, where Z is a random vector that is independent of Y. It follows that X + λU = g(Y) + Z + λU = g(Y) + Z′ where Z′= Z + λU. Because U is independent of X, Y and E, the variable Z′ will also be independent of Y. By property (2) of Theorem 1, one may easily conclude that the MEE estimator of X based on Y is identical to the MEE estimator of X + λU based on Y. This completes the proof.

Theorem 5: Let the random vector ( X Y ) has a joint (multivariate) Gaussian distribution, ( X Y ) ~ 𝒩 ( μ , Σ ), where μ = ( μ X μ Y ), Σ = ( Σ X X Σ X Y Σ Y X Σ Y Y ), then the MEE estimator of X based on Y will achieve the W-S lower bound, and it will be an affine linear function of Y.

Proof: It is easy to prove that the conditional distribution of X given Y has a Gaussian distribution with mean vector E [ X | Y ] = μ X + Σ X Y Σ Y Y 1 ( Y μ Y ) and covariance matrix Σ X X Σ X Y Σ Y Y 1 Σ Y X. That is, the conditional mean of X given Y is an affine linear function of Y, and the conditional covariance matrix of X given Y is constant (i.e., does not depend on Y). Since the shape of the Gaussian distribution depends only on the covariance matrix, the conditional density of X given Y has a fixed shape. And hence, the MEE estimator of X based on Y will achieve the W-S lower bound. By Theorem 3, the MEE estimator of X will also be an affine linear function of Y.

Remark: If X and Y are joint Gaussian, the MEE estimator of X based on Y will achieve the W-S lower bound. However, it should be noted that for most cases, the MEE estimator cannot achieve this performance bound. A simple example is given below.

Example 1: Consider the joint PDF pXY(x, y) = pX|Y (x|y)pY(y), where:

{ p Y ( y ) = 1 2 π exp ( y 2 2 ) p X | Y ( x | y ) = 1 2 π σ y exp ( x 2 2 σ y 2 )
where σy = 1 + αsin(y), |α| < 1. The joint density pXY is joint Gaussian only when α = 0, and in this case σy is independent of Y. For any y and α (|α|< 1), the conditional distribution of X given Y = y is Gaussian, which is symmetric and unimodal (SUM). According to the Theorem 1 in [12], the MEE estimator of X based on Y will be the conditional median of X given Y. For different α values, we can calculate the minimum error entropy and the W-S lower bound H(X|Y). The results are shown in Figure 1. As one can see clearly, when α ≠ 0 (joint non-Gaussian), the minimum error entropy is always above the W-S lower bound.

3. Characterization of the Gaussian Distribution

The W-S lower bound can be applied to characterize the Gaussian distribution, i.e., constructing some conditions under which a distribution is Gaussian (or joint Gaussian). The problem of characterization of the Gaussian distribution is an interesting problem, which has been extensively studied in the literature [2327]. First, we introduce a lemma.

Lemma 1 [24]: Let X and Y be two random vectors, X ∈ ℝp, Y ∈ ℝ q. If X𝒩(μX, ΣXX), and the distribution of Y given X is a q -dimensional Gaussian distribution with mean vector a + BX(a ∈ ℝq, B ∈ ℝ q×p), and constant covariance matrix Σ0 ∈ ℝq×q, then the joint distribution of ( X Y ) will be a (p + q)-dimensional Gaussian distribution, whose mean vector and covariance matrix are:

{ μ = ( μ X μ Y ) = ( μ X a + B μ X ) Σ = ( Σ X X Σ X Y Σ X Y T Σ Y Y ) = ( Σ X X Σ X X B T B Σ X X Σ 0 + B Σ X X B T )

Based on Lemma 1, we can state the following theorem.

Theorem 6: Let X and Y be two random vectors, X ∈ ℝn, Y ∈ ℝm. Assume that Y𝒩(μY, ΣYY), and the distribution of X given Y is a n -dimensional Gaussian distribution. If there exists a linear estimator = BY such that the error entropy H(E) achieves the W-S lower bound H (X|Y), where B∈ ℝ n×m, E = XBY, then ( X Y ) will be a (n + m)-dimensional multivariate Gaussian random vector.

Proof: Since the linear estimator = BY achieves the W-S lower bound, by Theorem 1, we have:

X = B Y + Z
where Z ∈ ℝn is a random vector that is independent of Y. And hence, the conditional mean vector of X givenY is a + BY, where a = E [Z]. In addition, the conditional covariance matrix of X given Y is equal to the covariance matrix of Z, which is a constant matrix (i.e., independent of Y). By applying Lemma 1, we complete the proof.

The next lemma is needed in the proof of Theorem 7.

Lemma 2 [25]: Let X ∈ ℝn and Y ∈ ℝm be two random vectors. Suppose the conditional densities pX|Y (x|y) and pY|X (y|x) are both (multivariate) Gaussian. Denote the covariance matrix of the conditional density pX|Y (x|y) by ΣX|Y (y). Then the following two statements are equivalent:

(i)

the joint density function pXY (x,y) is multivariate Gaussian;

(ii)

ΣX|Y(y) is constant on ℝm.

Theorem 7: Let X ∈ ℝn and Y ∈ ℝm be two random vectors. Suppose the conditional densities are pX|Y (x|y) and pY|X (y|x) both (multivariate) Gaussian. If the MEE estimator of X based on Y achieves the W-S lower bound H(X|Y), then the joint density function pXY (x,y) will be multivariate Gaussian.

Proof: Since the MEE estimator of X based on Y achieves the W-S lower bound, by Theorem 2, the conditional covariance matrix Σ X|Y (y) will not depend on y (i.e., be constant on ℝm). By Lemma 2, the joint density pXY will be multivariate Gaussian.

Before presenting Theorem 8, we introduce the third lemma, which is an extended version of Ghurye and Olkin’s theorem [23,26].

Lemma 3 [26]: Let U1 ∈ ℝp and U2 ∈ ℝq be two independent non-degenerate random vectors, and let X1 ∈ ℝp and X2 ∈ ℝq be two independent random vectors such that:

{ X 1 = A 1 U 1 + A 2 U 2 X 2 = C 1 U 1 + C 2 U 2
where A1 ∈ ℝp×p, A2 ∈ ℝp×q, C1 ∈ ℝq×p, C2 ∈ ℝq×q, among which A1 and C2 are square nonsingular matrices. Then U1 and U2 are multivariate Gaussian random vectors if the following conditions on A1, A2, C1 and C2 are satisfies:
(i)

P = C 2 C 1 A 1 1 A 2 is nonsingular,

(ii)

none of the rows of Γ 1 = A 1 1 A 2 and Γ2 = P−1C1 are null vectors.

Consider now the problem of estimating the source X ∈ ℝn given the observation Y = AX + Z, where Y ∈ ℝm, A ∈ ℝm×n, and Z ∈ ℝm is the additive noise that is independent of X, as shown in Figure 2.

For the above estimation problem, the following theorem holds:

Theorem 8: For the estimation problem in Figure 2, if there exists a linear estimator = BY such that the error entrpy H(E) achieves the W-S lower bound H(X|Y), where B ∈ ℝ n×m, E = XBY, then X and Z are multivariate Gaussian random vectors if the following conditions on A and B are satisfied:

(i)

the matrices InBA and Im + A(InBA)−1B are nonsingular, where In and Im are identity matrices of order n and m respectively,

(ii)

none of the rows of (InBA)−1B and [Im + A(InBA)−1B]−1A are null vectors.

Proof: Since Y = AX + Z, the error E can be expressed as:

E = X B Y = X B ( A X + Z ) = ( I n B A ) X B Z
Thus we have:
{ E = A 1 X + A 2 Z Y = C 1 X + C 2 Z
where A1 = InBA, A2 = − B, C1= A, C2 = Im. On the other hand, the error entropy H(E) achieves the W-S lower bound H(X|Y) if and only if E is independent of the observation Y. Then by applying Lemma 3, we arrive easily at the results. When m = n = 1, the conditions (i) and (ii) in Theorem 8 will be equivalent to the condition that A ≠ 0, B ≠ 0, and B ≠ 1/ A. Therefore, we have the following corollary:

Corollary 2: For the estimation problem in Figure 2, if X, Y, Z are all scalar random variables (m = n =1), and there exists a linear estimator = BY such that the error entropy H (E) achieves the W-S lower bound H (X|Y), where B ∈ ℝ, E = XBY, then X and Z are scalar Gaussian random variables if A ≠ 0, B ≠ 0, and B ≠ 1/ A.

Remark: It is worth noting that when the condition that A ≠ 0, B ≠ 0, and B ≠ 1/A does not hold, some of the variables in the estimating system will become degenerate random variables. Specifically, when A = 0, the variable AX will be a degenerate random variable; when B = 0, the estimator = BY will become a degenerate random variable; when B=1/A(A ≠ 0), the variable (1−BA)X will be a degenerate random variable.

4. Conclusions

MEE estimation provides an appealing approach to design optimal estimators in the framework of information theory. Recent successful applications of the MEE criterion in the areas of signal processing and machine learning suggest that this estimation method has significant potential advantages over traditional MMSE estimation, especially when data possess non-Gaussian distributions. Though it has shown remarkable success in many applications, some theoretical aspects of the MEE estimation need further study. In this work, we reexamine the W-S lower bound on the MEE estimation, which is a Shannon theory-type performance bound on the estimation error entropy. We present some basic properties of the W-S lower bound, and give some interesting results on the characterization of Gaussian distribution using this performance bound. It is hoped that the results of this work will help us to gain insights into the attainment of the W-S lower bound in MEE estimation.

Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 61372152, No. 90920301) and 973 Program (No. 2012CB316400, No. 2012CB316402).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Kailath, T.; Sayed, A.H.; Hassibi, B. Linear Estimation; Prentice Hall: Upper Saddle River, NJ, USA, 2000. [Google Scholar]
  2. Weidemann, H.L.; Stear, E.B. Entropy analysis of estimating systems. IEEE T. Inform. Theory 1970, 16, 264–270. [Google Scholar]
  3. Tomita, Y.; Ohmatsu, S.; Soeda, T. An application of the information theory to estimation problems. Inform. Contr 1976, 32, 101–111. [Google Scholar]
  4. Kalata, P.; Priemer, R. Linear prediction, filtering and smoothing: An information theoretic approach. Inform. Sci 1979, 17, 1–14. [Google Scholar]
  5. Minamide, N. An extension of the entropy theorem for parameter estimation. Inform. Contr 1982, 53, 81–90. [Google Scholar]
  6. Janzura, M.; Koski, T.; Otahal, A. Minimum entropy of error principle in estimation. Inform. Sci 1994, 79, 123–144. [Google Scholar]
  7. Wolsztynski, E.; Thierry, E.; Pronzato, L. Minimum-entropy estimation in semi-parametric models. Signal Process 2005, 85, 937–949. [Google Scholar]
  8. Principe, J.C. Information Theoretic Learning: Renyi’s Entropy and Kernel Perspectives; Springer: New York, NY, USA, 2010. [Google Scholar]
  9. Chen, B.; Zhu, Y.; Hu, J.; Principe, J.C. System Parameter Identification: Information Criteria and Algorithms; Elsevier Inc: London, UK, 2013. [Google Scholar]
  10. Cover, T.M.; Thomas, J.A. Element of Information Theory; Wiley & Sons Inc.: New York, NY, USA, 1991. [Google Scholar]
  11. Otahal, A. Minimum entropy of error estimate for multi-dimensional parameter and finite-state-space observations. Kybernetika 1995, 31, 331–335. [Google Scholar]
  12. Chen, T.-L.; Geman, S. On the minimum entropy of a mixture of unimodal and symmetric distributions. IEEE Trans. Inform. Theory 2008, 54, 3166–3174. [Google Scholar]
  13. Chen, B.; Zhu, Y.; Hu, J.; Zhang, M. On optimal estimations with minimum error entropy criterion. J. Franklin Inst 2010, 347, 545–558. [Google Scholar]
  14. Chen, B.; Zhu, Y.; Hu, J.; Zhang, M. A new interpretation on the MMSE as a robust MEE criterion. Signal Process 2010, 90, 3313–3316. [Google Scholar]
  15. Chen, B.; Principe, J.C. Some further results on the minimum error entropy estimation. Entropy 2012, 14, 966–977. [Google Scholar]
  16. Chen, B.; Principe, J.C. On the smoothed minimum error entropy criterion. Entropy 2012, 14, 2311–2323. [Google Scholar]
  17. Erdogmus, D.; Principe, J.C. An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems. IEEE Trans. Signal Process 2002, 50, 1780–1786. [Google Scholar]
  18. Erdogmus, D.; Principe, J.C. Generalized information potential criterion for adaptive system training. IEEE Trans. Neural Network 2002, 13, 1035–1044. [Google Scholar]
  19. Erdogmus, D.; Principe, J.C. Convergence properties and data efficiency of the minimum error entropy criterion in Adaline training. IEEE Trans. Signal Process 2003, 51, 1966–1978. [Google Scholar]
  20. Erdogmus, D.; Principe, J.C. From linear adaptive filtering to nonlinear information processing—The design and analysis of information processing systems. IEEE Signal Process. Mag 2006, 23, 14–33. [Google Scholar]
  21. Santamaria, I.; Erdogmus, D.; Principe, J.C. Entropy minimization for supervised digital communications channel equalization. IEEE Trans. Signal Process 2002, 50, 1184–1192. [Google Scholar]
  22. Chen, B.; Hu, J.; Pu, L.; Sun, Z. Stochastic gradient algorithm under (h, ϕ)-entropy criterion. Circuits Syst. Signal Process 2007, 26, 941–960. [Google Scholar]
  23. Ghurye, S.G.; Olkin, I. A characterization of the multivariate normal distribution. Ann. Math. Stat 1962, 33, 533–541. [Google Scholar]
  24. Fidalgo, J.L.; Albajar, R.A. Characterizing the general multivariate normal distribution through the conditional distributions. Extr. Math 1997, 12, 15–18. [Google Scholar]
  25. Bischoff, W.; Fieger, W. Characterization of the multivariate normal distribution by conditional normal distributions. Metrika 1991, 38, 239–248. [Google Scholar]
  26. Fisk, P.R. A note on a characterization of the multivariate normal distribution. Ann. Math. Stat 1970, 41, 486–494. [Google Scholar]
  27. Hamedani, G.G. On a recent characterization of the bivariate normal distribution. Metrika 1991, 38, 255–258. [Google Scholar]
Figure 1. The minimum error entropy and the W-S lower bound.
Figure 1. The minimum error entropy and the W-S lower bound.
Entropy 16 00814f1 1024
Figure 2. General setup of the source estimating problem.
Figure 2. General setup of the source estimating problem.
Entropy 16 00814f2 1024

Share and Cite

MDPI and ACS Style

Chen, B.; Wang, G.; Zheng, N.; Principe, J.C. A Note on the W-S Lower Bound of the MEE Estimation. Entropy 2014, 16, 814-824. https://doi.org/10.3390/e16020814

AMA Style

Chen B, Wang G, Zheng N, Principe JC. A Note on the W-S Lower Bound of the MEE Estimation. Entropy. 2014; 16(2):814-824. https://doi.org/10.3390/e16020814

Chicago/Turabian Style

Chen, Badong, Guangmin Wang, Nanning Zheng, and Jose C. Principe. 2014. "A Note on the W-S Lower Bound of the MEE Estimation" Entropy 16, no. 2: 814-824. https://doi.org/10.3390/e16020814

Article Metrics

Back to TopTop