1. Introduction
A population quantity (for example, the average height of all men) can be estimated in many different ways. An unbiased estimator (see Definition 1) is one that provides zero bias (that is, it estimates the population quantity with zero bias). Some examples of unbiased estimators are estimation for average length of stay in intensive care units in the COVID-19 pandemic (Lapidus et al. [
1]); estimation of cumulative incidence incorporating antibody kinetics and epidemic recency (Takahashi et al. [
2]); estimation of background distribution for automated quantitative imaging (Silberberg and Grecco [
3]); and estimation of target tracking in Doppler radar (Han et al. [
4]). The Rao–Blackwell theorem is a generator for unbiased estimators with small variances. In this note, we aim at generalizing this effective theorem for finding those estimators. Fisher [
5] introduced sufficient statistics in 1920; see Definition 2.
Definition 1. Let , where are random variables from an unknown population , . Let be a parameter taking real values. An estimator , , of is unbiased if and only if for every .
Definition 2 (Sufficient statistic (Fisher [
5])).
Let , where are random variables from an unknown population , . A statistic is said to be sufficient for θ if the conditional probability distribution of X, given the statistic , is independent of θ. Examples of sufficient statistics can be found in the books Lehmann [
6], Casella and Berger [
7], and Shao [
8]. We should understand sufficient statistics better to derive uniformly minimum variance unbiased estimators (UMVUEs); see Definition 4. Statisticians have cleverly embedded sufficient statistics into estimators, which is the main idea of the Rao–Blackwell theorem; see Rao [
9] and Blackwell [
10]. UMVUEs can be calculated by complete sufficient statistics (see Definition 5), leading to the Lehmann–Scheffé theorem; see Lehmann and Scheffé [
11,
12] and Kumar and Vaish [
13]. A complete statistic is defined by Definition 3.
Definition 3. Let , where are random variables from an unknown population . A statistic , , is said to be complete for if and only if, for any Borel-measurable function f from to , for all implies almost surely .
Definition 4. Let , where are random variables from an unknown population . An unbiased estimator of is a UMVUE if and only if Var Var for every and every unbiased estimator of .
Definition 5 (Complete sufficient statistic (Fisher [
5])).
Let be a sufficient statistic for θ. If with probability 1, for some function g, then it is said to be a complete sufficient statistic for θ. Applications of the Rao–Blackwell and Lehmann–Scheffé theorems are still widespread. Application areas have included reliability estimation (Kumar and Vaish [
13]), adaptive cluster sampling (Felix-Medina [
14]), alchemical free energy calculation (Ding et al. [
15]), hazardous source parameter estimation (Ristic et al. [
16]) and quantum probability (Sinha [
17]).
However, nonconstant functions can be UMVUEs whenever there is not a complete sufficient statistic. Some authors try solving these problems by Theorem 1 by focusing on problem 5.11 in Rao [
9] or pages 76–77 in Lehmann and Scheffé [
11,
12] (which are essentially Example 7 below).
Theorem 1 (Lehmann–Scheffé theorem (Lehmann and Scheffé [
11,
12])).
Let , where are random variables from an unknown population , . If and only if condition for a statistic to be UMVUE of its mean is that for all and all , where denotes the set of all the unbiased estimators of 0. This theorem can be used whenever there are no complete sufficient statistics. It is a competitor to the Rao–Blackwell theorem.
This fact is hardly pointed out or explained in undergraduate or graduate textbooks; see, for example, Bondesson [
18]. The motivation to introduce a new concept of sufficient statistic called an
-sufficient statistic comes from the above discussion. We investigate the properties of
-sufficient statistics and compare them with those of sufficient statistics. Then, the Rao–Blackwell theorem (RBT) and Lehmann–Scheffé theorem (LST) will be generalized in a way that can solve some of the problems where UMVUEs exist but there are no complete sufficient statistics; cf. problem 5.11 in Rao [
19], pages 76–77 in Lehmann [
6], page 167, Example 3.7 in Shao [
8], page 366, Example 10 in Rohatgi and Ehsanes [
20], page 377, Section 7.6.1 in Mukhopadhyay [
21], page 243 in Peña and Rohatgi [
22], page 293, Section 12.4 in Roussas [
23] and pages 330–331 in Mood et al. [
24]. Some of the theorems are restated and proved by using the newly introduced
-sufficient statistic.
Definition 6 (Ancillary statistic). Let , where are random variables from an unknown population , . A statistic is said to be ancillary for θ if its distribution is the same for all .
Boos and Hughes-Oliver [
25] state that “If a minimal sufficient statistic is not complete, then by the suggestion of Fisherian tradition we should consider condition on ancillary statistics (see Definition 6) for the purposes of inference. This approach runs into problems because there are many situations where several ancillary statistics exists but there are no maximal “ancillaries”. Of course, when a complete sufficient statistic exists, Basu’s theorem assures us that we need not worry about conditioning on ancillary statistics since they are all independent of the complete sufficient statistic”. We suggest complete
-sufficient statistics for the purposes of inference when there are no complete sufficient statistics. Theorem 1 assures that we need not worry about ancillary statistics since they are uncorrelated regarding complete
-sufficient statistics.
2. The Main Contribution
If the minimal sufficient statistic is not complete, then the RBT and LST will not be of much use, as has been explicitly stated in various books and papers; see, for example, page 46, Section 2, Example 1 of Bondesson [
18], page 243 of Peña and Rohatgi [
22] page 293, Section 12.4 of Roussas [
23], pages 330–331 of Mood et al. [
24], page 343, Section 7.3 of Casella and Berger [
7], page 86, Example 1.8 of Lehmann and Casella [
26], Section 1 of Bahadur [
27] and Section 1 of Stigler [
28].
The main contribution of this note is a generalization of the RBT and LST, resulting in the use of the newly introduced -sufficient statistics. This enables us to obtain UMVUEs even when the minimal sufficient statistic is not complete, in which case the RBT and LST are not directly applicable.
Consider a model . Let denote the set of probability measures on the sample space . Let denote an element in . is the population. is a sample. Let ⟶, and denote, respectively, the identity mapping, a measurable space and a measurable mapping (that is, for all ). is a statistic to , written as . is referred to as a U-estimable parameter if is an unbiased estimator.
Throughout this note, we assume that
, where
are random variables from an unknown population
. Assume also
has an unbiased estimator. Let
denote the class of unbiased estimators
for
. All the considered estimators are assumed to have finite variances. The space used in this note is
and the elements of
are Borel sets. For related notation and discussions, see Shao [
8].
3. Sufficient Statistics
Sufficient statistics can be used to derive maximum likelihood estimators of a population quantity. Maximum likelihood estimation is a popular method for estimation, so sufficient statistics are important. Sufficient statistics were defined in Definition 2. Two weaker concepts of sufficiency, which are tailored to a given unbiased estimable aspect , are introduced and discussed in the following. Some properties of these statistics are studied in the sequel.
3.1. -Sufficient Statistic in Distribution
Definition 7. Let , where are random variables from an unknown population . A statistic is -sufficient in distribution for if, for all , there is a Markov kernel such that, for every , is a version of a regular conditional distribution of given under .
Definition 7 introduces a class of statistics that are weaker than sufficient statistics, which is not the main aim of this note. These statistics could be used in Rao–Blackwell and Lehmann–Scheffé theorems. We use this idea in Definition 8.
Example 1 (Example of Meeden [
29]).
Let X be Poisson-distributed with , so X belongs to the exponential family. Then, X is a complete sufficient statistic and is only unbiased estimator for . By Definition 7, is an -sufficient statistic in distribution for for k a constant. We can check that is a UMVUE for . The estimator could not be suitable for in the same way that in the Bernoulli distribution with parameter p the estimator X will not be suitable. Of course, increasing the sample size or varying the loss function remedies this deficiency.
3.2. -Sufficient Statistic
To derive UMVUEs when there are no complete sufficient statistics, we need to introduce a new concept named an -sufficient statistic for . It is defined as follows.
Definition 8. Let , where are random variables from an unknown population . A statistic is an -sufficient statistic for if, for all , there is a measurable mapping such that for every we have almost surely .
Example 2. Let X from have a discrete distribution withwhere is unknown. is an -sufficient statistic for becausealmost surely for every and . The expectations needed for the left hand side of (1) areand X is a minimal sufficient statistic for because of the LST. However, X is not complete since . Also, is not an -sufficient statistic in distribution for since its conditional distribution depends on θ.
For having all the unbiased estimators of , see the following proof.
For every , we have Then, for any , Comparing power series coefficients, we haveorwhere . Some properties of -sufficient statistics are in Theorem 2.
Theorem 2. Let . Consider
- (i)
a sufficient statistic for (or θ),
- (ii)
an -sufficient statistic in distribution for ,
- (iii)
an -sufficient statistic for .
Then, we have
- (a)
any sufficient statistic for is an -sufficient statistic in distribution for ;
- (b)
any -sufficient statistic in distribution for is an -sufficient statistic for ;
- (c)
any sufficient statistic for is an -sufficient statistic for .
Proof. (a) follows because the conditional distribution of samples given a sufficient statistic does not depend on . (b) follows because the conditional distribution of unbiased estimators given an -sufficient statistic does not depend on . (c) follows because the conditional distribution of samples given a sufficient statistic does not depend on . □
Remark 1. In general, the converse of none of the three parts of Theorem 2 holds (see Examples 1 and 2).
It is clear from Theorem 2 and Remark 1 that the class of -sufficient statistics for contains sufficient statistics for . Also, we can conclude from Theorem 2 that the jointly sufficient statistics are -sufficient statistics.
Proposition 1. Let , where are random variables from an unknown population . If an unbiased estimator is unique for , then is an -sufficient statistic for .
Proof. Obviously,
almost surely
because of the definition of
-sufficiency; cf. Casella and Berger [
7] and Shao [
8]. □
Proposition 2. Let , where are random variables from an unknown population . Let be an -sufficient statistic for such that for , another statistic, and g, a one-to-one measurable function. Then, is an -sufficient statistic for .
Proof. Let
denote an unbiased estimator of
. Then, we have
=
almost surely
, which shows that
is independent of
; cf. Casella and Berger [
7]. □
Remark 2. Let , where are random variables from an unknown population . Let be an -sufficient statistic for and another statistic such that for a measurable function g. We expect to be an -sufficient statistic for , but, actually, it is not. Consider Example 2 again: Let and , 0 and 2 for and , respectively. Then, verify that (i) is an -sufficient statistic, (ii) is a function of but (iii) is not an -sufficient statistic.
4. A Generalization of RBT and LST
We now apply the RBT for arbitrary -sufficient statistics for to obtain a better estimator.
Theorem 3. Let , where are random variables from an unknown population . Let be an -sufficient statistic for . Let be an unbiased estimator of a U-estimable , and the loss function be a strictly convex function of . Then, if has finite expectation and risk, we have , and, if , then the risk of the estimator satisfies unless almost surely .
Proof. Since
L is convex, by Jensen’s inequality,
and
Hence, the result follows from Definition 8; see Lehmann and Casella [
26] for details. □
We now reexpress Lemma 1.10 in Lehmann and Casella [
26] within the new framework.
Lemma 1. Let , where are random variables from an unknown population . Let be a complete -sufficient statistic for . Then, every U-estimable has one and only one unbiased estimator that is a function of . Of course, uniqueness here means that any two such functions agree almost surely .
Proof. If
is another unbiased estimator, then
. By the completeness property,
with probability one; see Lehmann and Casella [
26] for details. □
The generalization of LST (Lehmann and Scheffé [
11], Theorem 5.1) by using a complete
-sufficient statistic for
is as follows.
Theorem 4. Let , where are random variables from an unknown population . Suppose that is a complete -sufficient statistic for . Then, we have the following:
- (i)
For every U-estimable , there exists an unbiased estimator that uniformly minimizes the risk for any loss function that is convex in δ; therefore, the estimator is UMVUE of .
- (ii)
The UMVU estimator of (i) is a unique unbiased estimator and is a function of ; it has minimum risk, provided its risk is finite and is strictly convex in δ.
Proof. (i) If U is unbiased, by Theorem 3, we can consider the estimator of whose risk is less than the risk of U. (ii) If is another estimator with minimum risk, then must have less risk by Theorem 3, which would be impossible. Thus, by Lemma 1, . □
Theorem 5. Let , where are random variables from an unknown population , . Let be an unbiased estimator for and an -sufficient statistic for such that for a measurable function g. Then, if and only if condition for to be a UMVUE of is that for all and , where denotes the set of all unbiased estimators of 0.
Proof. Suppose that
. The result follows because of
∈
and
where
is an unbiased estimator of 0.
is a statistic since
{
−
∣
} is independent of
. The converse follows because of
□
Theorem 6. Let , where are random variables from an unknown population , . Let be an -sufficient statistic for . In addition, suppose for every unbiased estimator for there is a measurable function g such that . Then, is a UMVUE if almost surely for every and .
Proof. For , we have = = 0 since = 0 almost surely . Since , is a UMVUE by the LST. □
5. Complete -Sufficient Statistic
We are interested in finding an -sufficient statistic with the simplest structure. A minimal -sufficient statistic is an -sufficient statistic that is a function of any other -sufficient statistic.
Definition 9 (Minimal -sufficient statistics). Let , where are random variables from an unknown population . Let be an -sufficient statistic for . A statistic is a minimal -sufficient statistic for if and only if, for any other statistic that is an -sufficient for , there exists a measurable function ψ such that almost surely .
Theorem 7. Let , where are random variables from an unknown population , . Let be a complete sufficient statistic for (or θ) such that , , has mean . Then, any -sufficient statistic for is a sufficient statistic for (or θ).
Proof. Let be an -sufficient statistic for . By Theorem 3, var ≤ var. Since is a UMVUE, almost surely because there can be no better estimators. So, we can find a measurable function g such that almost surely . Hence, is a sufficient statistic. □
Thus, we can apply
-sufficient statistics for
in case complete sufficient statistics do not exist. Intuitively, an
-sufficient statistic with the complete property will be a minimal
-sufficient statistic. The following theorem, a version of Bahadur’s theorem, see Bahadur [
27], states an important property of minimal
-sufficient statistics.
Theorem 8. Let , where are random variables from an unknown population . If , , is a complete -sufficient statistic for , then is a minimal -sufficient statistic for .
Proof. Let be an -sufficient statistic for . By Theorem 3, almost surely since is a UMVUE. □
We now show that complete -sufficient statistics may not exist.
Example 3 (Complete
-sufficient statistics may not exist).
Let X be a random variable with , and then X is not complete. , are all of the zero unbiased estimators. Since X is sufficient, X is an -sufficient statistic for θ (see Theorem 2, part a). However, a complete -sufficient statistic for θ does not exist. Otherwise, for every and some , we would have almost surely , where is assumed to be a complete -sufficient statistic for θ, but this cannot hold since is not UMVUE for θ. Sincethere is no such that . , are all of the unbiased estimators. 7. Conclusions
Sufficient statistics are of central concern for statisticians. They play a fundamental role in Rao–Blackwell and Lehmann–Scheffé theorems. By Theorem 3, every sufficient statistic is an -sufficient statistic. The class of -sufficient statistics contains all of the sufficient statistics and also some statistics that are not necessarily sufficient. So, the factorization theorem, and its corollaries, should not hold generally for -sufficient statistics. The concepts closest to -sufficient statistics are those of “partial sufficient” and “sufficient subspace”. However, they are slightly different.
When a complete sufficient statistic is lacking, there may sometimes be nonconstant parametric functions that can be UMVU-estimated. This fact is seldom pointed out and exemplified in undergraduate and graduate textbooks. In this note, we have shown how the concept of -sufficient statistics can be used to obtain UMVUEs in these contexts.
More research based on the concept of -sufficiency are under investigation. They are