Some More Results on Characterization of the Exponential and Related Distributions

Abstract

There are given characterizations of the exponential distribution based on the properties of independence of linear forms with random coefficients. Results based on the constancy of regression of one statistic in a linear form are obtained. Related characterizations based on the property of the identical distribution of statistics are also provided.

1. Introduction

29 May 2023 marks the 95th anniversary of the birth of Abram Aronovich Zinger (1928–2019). One of his main areas of scientific interest was the theory of characterizations of probability distributions. He published a number of outstanding results in this field (see [1,2,3,4]). Many of these results are connected to the characterization of the normal law by the independence and/or identical distribution property of suitable statistics. In cooperation with Professor Yuri Vladimirovich Linnik, he was the first who provided such characterization using linear forms with random coefficients [4]. However, in our discussions with Professor Zinger, he expressed the opinion that slightly different properties may characterize other classes of distributions. Now, we are talking about the independence properties of linear forms with random coefficients because characterizations of different probability distribution classes by the independence of non-linear statistics have been known for a very long time (see, for example, [2,5,6]). Some characterizations of identical distribution properties and the constancy of regression are known as well [3,7]. In three publications ([8,9]), and Ref. [10], the authors attempted to show that Professor Zinger’s opinion on the possibility of using the independence of linear forms with random coefficients for the characterization of non-normal distribution is true. Namely, such properties are suitable for the characterization of two-point and hyperbolic secant distributions. The aim of this paper is to show that the exponential distribution may be characterized in a similar way.

Several thousand papers are devoted to characterizations of the exponential and related laws. I neither have the opportunity nor the plan to attempt to review them. However, let me mention two books on earlier characterizations of this kind. These are monographs [11,12]. Classical results presented in these books are mostly related to various variants of the lack-of-memory property, properties of order statistics, and independence of non-linear statistics. This trend continues in subsequent publications on the characteristic properties of the exponential distribution. Let us mention that the independence property of liner forms with constant coefficients is a characteristic property of the Gaussian distribution. The results given here show that the presence of randomness in the coefficients of forms changes the situation radically. In our opinion, the randomness appears in many practical problems. Therefore, the exceptional role of the Gaussian distribution in applied problems is significantly exaggerated. The limit theorems for sums of a random number of random variables also speak about this. Therefore, the appearance of a normal law is not the rule, but the exception.

The results of the paper were partially announced in the pre-print [13].

2. Exponential Distribution and Linear Form with Random Coefficients: One-Dimensional Case

Let us start with a characterization of the exponential distribution by the independence property.

Theorem 1. 

Let ${\epsilon }_p$ be a random variable taking values 1 with probability $p\in (0,1)$ and 0 with probability $1-p$. Suppose that $X,Y$ are independently identically distributed (i.i.d.) random variables, positive almost surely (a.s.), and independent with ${\epsilon }_p$. Consider linear forms

$$L_1=(1-p)aX+{\epsilon }_{p}aY\mathrm{and}{L}_2=pbX+(1-{\epsilon }_p)bY,$$

(1)

where $a,b$ are positive constants. Then the forms $L_1$ and $L_2$ are independent if and only if X and Y have exponential distributions.

Proof. 

The Laplace transform of the random vector $(L_1,L_2)$ has the following form

$$\mathrm{I}\mathrm{E}exp\{-s{L}_1-t{L}_2\}=f(a(1-p)s+bpt)\left(f\left(as\right)p+f\left(bt\right)(1-p)\right),$$

(2)

where f is the Laplace transform of the random variable X. Random forms $L_1$ and $L_2$ are independent if and only if

$$\mathrm{I}\mathrm{E}exp\{-s{L}_1-t{L}_2\}=\mathrm{I}\mathrm{E}exp\{-s{L}_1\}\mathrm{I}\mathrm{E}exp\{-t{L}_2\},$$

which is equivalent to

$$f(a(1-p)s+bpt)\left(f\left(as\right)p+f\left(bt\right)(1-p)\right)=$$

$$f\left(a(1-p)s\right)\left(f\left(as\right)p+(1-p)\right)f\left(bpt\right)\left(p+f\left(bt\right)(1-p)\right).$$

(3)

We change $as$ and $bt$ by new variables, where we denote s and t again. Instead of (3), we obtain

$$f((1-p)s+pt)\left(f\left(s\right)p+f\left(b\right)(1-p)\right)=$$

$$f\left((1-p)s\right)\left(f\left(s\right)p+(1-p)\right)f\left(pt\right)\left(p+f\left(t\right)(1-p)\right).$$

(4)

Substituting $f\left(s\right)=1/(1+\lambda s)$ ($\lambda >0$) into (4), we find that the Laplace transform of the exponential distribution satisfies this equation. In other words, $L_1$ and $L_2$ are independent for exponentially distributed X and Y.

Let us show the inverse statement: the independence of $L_1$ and $L_2$ implies that X and Y have exponential distribution. To this aim, we put $t=s$ into (4). We obtain

$$f^2\left(s\right)=f\left(ps\right)f\left((1-p)s\right)\left(p(1-p)f^2\left(s\right)+\left(p^2+{(1-p)}^2\right)f\left(s\right)+p(1-p)\right).$$

(5)

Now, we want to show that Equation (5) has no other solutions that correspond to the Laplace transform of a probability distribution, other than $\phi (s;\lambda )=1/(1+\lambda s)$ ($\lambda >0$). The method of intensely monotone operators is suitable for this purpose [14]. However, Equation (5) does not have a convenient form for applying any theorem from [14]. We apply the method directly. It is clear that

• $\phi (s;\lambda )$ satisfies (5) for any $\lambda >0$;
• $\phi (s;\lambda )$ is analytic in s in some neighborhood of the point $s=0$;
• For any positive $s_o$, there is ${\lambda }_o$, such that $f\left(s_o\right)=\phi (s_o,{\lambda }_o)$, where f is a fixed solution of (5).

Let us consider the difference between $f\left(s\right)$ and $\phi (s,{\lambda }_o)$. From (3), it follows that $\phi (s_o;{\lambda }_o)=f\left(s_o\right)$. We define the set $S=\{s:0<s\le s_o,f\left(s\right)=\phi (s,{\lambda }_o)\}$ and denote $s^{*}=infS$. Because both $f\left(s\right)$ and $\phi (s,{\lambda }_o)$ are continuous, we have $f\left(s^{*}\right)=\phi (s^{*},{\lambda }_o)$. We show that the case $s^{*}>0$ is impossible. In this case, we would have

$$f^2\left(s^{*}\right)=f\left(p{s}^{*}\right)f\left((1-p)s^{*}\right)\left(p(1-p)f^2\left(s^{*}\right)+\left(p^2+{(1-p)}^2\right)f\left(s^{*}\right)+p(1-p)\right)$$

and

$${\phi }^2\left(s^{*}\right)=\phi (p{s}^{*},{\lambda }_o)\phi ((1-p)s^{*},{\lambda }_o)(p(1-p){\phi }^2(s^{*},{\lambda }_o)+$$

$$\left(p^2+{(1-p)}^2\right)\phi (s^{*},{\lambda }_o)+p(1-p)).$$

We have $f\left(s^{*}\right)=\phi (s^{*},{\lambda }_o)$, and two previous relations give us

$$f\left(p{s}^{*}\right)f\left((1-p)s^{*}\right)=\phi (p{s}^{*},{\lambda }_o)\phi ((1-p)s^{*},{\lambda }_o).$$

(6)

However, $f\left(s\right)\ne \phi (s,{\lambda }_o)$ for all $s\in (0,s^{*})$. Therefore, either $f\left(p{s}^{*}\right)f\left((1-p)s^{*}\right)>\phi (ps,{\lambda }_o)\phi ((1-p)s^{*},{\lambda }_o)$ or $f\left(p{s}^{*}\right)f\left((1-p)s^{*}\right)<\phi (p{s}^{*},{\lambda }_o)\phi ((1-p)s^{*},{\lambda }_o)$ in contradiction with (6). This leads us to the fact that $s^{*}=0$. The latest is possible only if there exists a sequence of pints $\{s_j,j=1,2,\dots \}$, such that $s_j\to 0$, as $j\to \infty$ and $f\left(s_j\right)=\phi (s_j,{\lambda }_o)$. It shows that $f\left(s\right)=\phi (s,{\lambda }_o)$ for all $s\ge 0$ (see Example 1.3.2 from [14]). □

Note The following highlights are the same. that under the conditions of Theorem 1, we have

$$((1-p)X+{\epsilon }_{p}Y,pX+(1-{\epsilon }_p)Y)\stackrel{d}{=}(X,Y)$$

(7)

if and only if X has an exponential distribution.

Theorem 1 implies that X has an exponential distribution. It is easy to verify that $X\stackrel{d}{=}(1-p)X+{\epsilon }_{p}Y$ for independent exponentially distributed X and Y. Relation (7) raises several important questions. Let us mention a few of them:

(1)

Is the property $aX+bY\stackrel{d}{=}\left((1-p)a+pb\right)X+\left(a{\epsilon }_p+b(1-{\epsilon }_p)\right)Y$ characteristic for an exponential distribution?

(2)

Let $X_1,\dots ,X_n,\dots$ be a sequence of independent random variables distributed identically with X. Suppose that $Y,{\epsilon }_p$ are as defined in Theorem 1 and $\{{\nu }_q,q\in (0,1)\}$ has geometric distribution with parameter q, independent of the sequence of $X_1,\dots ,X_n,\dots$. When does $(1-p)X+{\epsilon }_{p}Y\stackrel{d}{=}q{\sum }_{j=1}^{{\nu }_q}{X}_j$ hold?

(3)

Does the relation

$$\mathrm{I}\mathrm{E}\{pX+(1-{\epsilon }_p)Y|(1-p)X+{\epsilon }_{p}Y\}=\mathrm{const}$$

characterize an exponential distribution?

Below, we shall attempt to answer these questions for some particular cases.

(Let us start with question 1). Below we give two results in connection with this question.

Theorem 2. 

Let ${\epsilon }_p$ be a random variable, taking values 1 with probability $p\in (0,1)$ and 0 with that of $1-p$. Suppose that X and Y are independent and identically distributed (i.i.d.) random variables that are positive and independent with ${\epsilon }_p$. Linear forms X and $(1-p)X+{\epsilon }_{p}Y$ are identically distributed

$$X\stackrel{d}{=}(1-p)X+{\epsilon }_{p}Y$$

(8)

if and only if X has an exponential distribution.

Proof. 

The forms are identically distributed if and only if their Laplace transforms satisfy the equation

$$f\left(t\right)=f\left((1-p)t\right)\left(pf\left(t\right)+(1-p)\right)$$

or

$$f\left(t\right)=\frac{(1-p)f\left(\right(1-p\left)t\right)}{1-pf\left(\right(1-p\left)t\right)}.$$

Now, the result follows from [15,16]. □

Theorem 3. 

Let ${\epsilon }_p$ be a random variable taking values 1 with probability $p\in (0,1)$ and 0 with that of $1-p$. Suppose that X and Y are independent and identically distributed (i.i.d.) random variables that are positive and independent with ${\epsilon }_p$. Let $0<a<b<1$. We define $V=(b-a)/\left(pb+(1-p)a\right)$ and $k=[log(1/p)/log(V\left)\right]+2$, where square brackets are used for the integer parts of the numbers in them. Suppose that X has a finite moment of order k. Linear forms

$$aX+bY\stackrel{d}{=}\left((1-p)a+pb\right)X+\left(a{\epsilon }_p+b(1-{\epsilon }_p)\right)Y$$

(9)

are identically distributed if and only if X has an exponential distribution.

Proof. 

In Laplace transformation terms, the relation (9) takes the form

$$f\left(as\right)f\left(bs\right)=f\left(cs\right)\left(pf\left(as\right)+(1-p)f\left(bs\right)\right),c=(1-p)a+pb.$$

(10)

It is easy to verify that the Laplace transform of exponential distribution $g\left(s\right)=1/(1+\lambda s)$ is a solution of (10) for arbitrary $\lambda >0$. Therefore, (9) holds for exponential distribution with arbitrary scale parameters. We introduce new functions $\phi \left(s\right)=1/f\left(s\right)$ and $\psi \left(s\right)=1/g\left(s\right)$. It is clear that $\phi \left(0\right)=\psi \left(0\right)=1$ and

$$\phi \left(s\right)=p\phi \left(Bs\right)+(1-p)\phi \left(As\right),$$

(11)

where $A=a/c<1$ and $B=b/c>1$. Obviously, the function $\psi$ satisfies Equation (11) as well. Note that (11) is not a Cauchy equation because the numbers $a,b$, and c are fixed. The function $f\left(s\right)$ is the Laplace transform of a probability distribution that is not degenerate at zero. Therefore, $\phi \left(s\right)$ is greater or equal to 1 for all positive values of s and tends monotonically to infinity as $s\to \infty$. Because X has moments up to order k, the functions $f\left(s\right)$ and $\phi \left(s\right)$ are at least k times differentiable for $s\in [0,\infty )$. It is easy to see that $log(1/p)/log\left(V\right)>1$ and, therefore, $k\ge 3$. It is also clear that

$$p{B}^j+(1-p)A^j\ne 1$$

(12)

for $j=2,3,\dots$, while the left-hand side of (11) coincides with 1 for $j=0,1$. Therefore, the derivative ${\phi }^{\left(j\right)}\left(0\right)$ may be arbitrary for $j=0,1$ and it equals zero for $j=2,\dots ,k$ (to obtain this, it is sufficient to take the j-th derivative from both sides of (11), and setting $s=0$).

The function $f\left(s\right)$ is the Laplace transform of a non-degenerate at zero probability distribution and, therefore, $\phi \left(0\right)=1$ and ${\phi }^{\prime }\left(0\right)>0$. From (11), we have

$$\phi \left(Bs\right)=\frac{1}{p}\left(\phi \left(s\right)-(1-p)\phi \left(As\right)\right)<\frac{1}{p}\phi \left(s\right).$$

Therefore,

$$\phi \left(B^{m}s\right)<\frac{1}{p^m}\phi \left(s\right),m=1,2,\dots ,$$

and, consequently,

$$\phi \left(s\right)<C{s}^{\gamma }$$

for sufficiently large values of s. Here, $C>0$ is a constant and $\gamma =log(1/p)/log\left(B\right)$. The difference $\xi \left(s\right)=\phi \left(s\right)-\psi \left(s\right)$ satisfies Equation (11) and conditions:

(a) 

${\xi }^{\left(j\right)}\left(0\right)=0$ for $j=0,1,\dots k$;

(b) 

$\left|\xi \left(s\right)\right|<C{s}^{\gamma }$ for sufficiently large s.

We introduce a space $\mathfrak{F}$ of real continuous functions $\zeta \left(s\right)$ on $[0,\infty )$ for which the integral ${\int }_0^{\infty }\left|\zeta \left(s\right)\right|/s^{k+1}ds$ converges. According to properties $a)$ and $b)$, we have $\xi \in \mathfrak{F}$. Wer define a distance d on $\mathfrak{F}$ as

$$d({\zeta }_1,{\zeta }_2)={\int }_0^{\infty }\left|{\zeta }_1\left(s\right)-{\zeta }_2\left(s\right)\right|\frac{ds}{s^{k+1}}.$$

It is clear that $(\mathfrak{F},d)$ is a complete metric space. We introduce the following operator

$$\mathcal{A}\left(\zeta \right)=\frac{1}{p}\left(\zeta (s/B)-(1-p)\zeta (sA/B)\right)$$

from $\mathfrak{F}$ to $\mathfrak{F}$. For $\zeta ={\zeta }_1-{\zeta }_2$, where ${\zeta }_1,{\zeta }_2\in \mathfrak{F}$, we have

$$d(\mathcal{A}\left({\zeta }_1\right),\mathcal{A}\left({\zeta }_2\right))={\int }_0^{\infty }\left|\frac{1}{p}\left(\zeta (s/B)-(1-p)\zeta (sA/B)\right)\right|\frac{ds}{s^{k+1}}\le$$

$${\int }_0^{\infty }\left|\frac{1}{p}\zeta (s/B)\right|\frac{ds}{s^{k+1}}+{\int }_0^{\infty }\left|\frac{1}{p}(1-p)\zeta (sA/B)\right|\frac{ds}{s^{k+1}}\le$$

$$\frac{1+(1-p)A^k}{p{B}^k}d({\zeta }_1,{\zeta }_2)=\rho d({\zeta }_1,{\zeta }_2),$$

where

$$\rho =\frac{1+(1-p)A^k}{p{B}^k}<1.$$

Now, we see that $\mathcal{A}$ is the contraction operator. It has only one fixed point in the space $\mathfrak{F},d$. This point is $\zeta \left(s\right)=0$ for all $s\ge 0$. In other words, $\xi \left(s\right)=0$ for all $s\ge 0$ and $\phi =\psi$. □

Let us mention that Theorems 2 and 3 give particular answers for question 1) above.

(Let us now go to question 2). Namely, let $X_1,\dots ,X_n,\dots$ be a sequence of independent random variables distributed identically with X. Suppose that Y, and ${\epsilon }_p$ are from Theorem 1. Suppose that $\{{\nu }_q,q\in (0,1)\}$ has geometric distribution with parameter q independent of the sequence of $X_1,\dots ,X_n,\dots$. When does $(1-p)X+{\epsilon }_{p}Y\stackrel{d}{=}q{\sum }_{j=1}^{{\nu }_q}{X}_j$ hold? In terms of the Laplace transform, we have to solve the following equation

$$f\left((1-p)s\right)\left((1-p)+pf\left(s\right)\right)\left(1-(1-q)f\left(qs\right)\right)=qf\left(qs\right).$$

(13)

The case $q=1-p$ appears to be very simple.

Theorem 4. 

Under the conditions above, if $q=1-p$, Equation (13) holds if and only if $f\left(s\right)$ is the Laplace transform of an exponential distribution.

Proof. 

For this case, $q=1-p$ makes the following change of a function. We set $1/f\left(s\right)=1+\xi \left(s\right)$. After simple transformations, we come to

$$\xi \left(s\right)=\frac{1}{1-p}\xi \left((1-p)s\right).$$

The statement follows now from [14], similar to the proof of Theorem 2. □

Theorem 5. 

Under the conditions above, suppose the distribution of X has the moments of all orders. Let $q^{k-1}\ne p+{(1-p)}^k$ for all $k=2,3,\dots$. Equation (13) holds if and only if $f\left(s\right)$ is the Laplace transform of an exponential distribution.

Proof. 

Let us rewrite (13) in the form

$$f\left((1-p)s\right)\left((1-p)+pf\left(s\right)\right)\left(1-(1-q)f\left(qs\right)\right)-qf\left(qs\right)=0.$$

(14)

The random variable X has moments of all orders. Therefore, its Laplace transform is infinitely differentiable for all values of $s\ge 0$. We differentiate both sides of (14) k times with respect to s and put $s=0$. The coefficient at $f^k\left(0\right)$ is

$${(1-p)}^{k}q+qp-(1-q)q^k-q^{k+1}=q\left({(1-p)}^k+p-q^{k-1}\right)\ne 0$$

(15)

for $k=2,3,\dots$. For $k=1$, coefficient (15) is zero. This means the derivatives of f of order $k>1$ calculated at zero are uniquely defined by the value of the first derivative $f^{\prime }\left(0\right)<0$. However, the Laplace transform of the Exponential distribution satisfies (14), and its derivative at zero may be taken as an arbitrary negative number. □

(Let us now go to question 3).

Theorem 6. 

Let $X,Y$ be i.i.d. positive random variables possessing finite moments of all orders. Suppose that ${\epsilon }_p$ is a Bernoulli random variate independent of $X,Y$, and takes value 1 with probability p and 0 with $1-p$, $0<p<1$. The relation

$$\mathrm{I}\mathrm{E}\{pX+(1-{\epsilon }_p)Y|(1-p)X+{\epsilon }_{p}Y\}=\mathrm{const}$$

(16)

holds if and only if X has an exponential distribution.

Proof. 

Equation (16) may be written in terms of the Laplace transform, as

$$-p{f}^{\prime }\left((1-p)t\right)\left(pf\left(t\right)+(1-p)\right)=\lambda pf\left((1-p)t\right)f\left(t\right),$$

(17)

where $\lambda =\mathrm{I}\mathrm{E}X>0$. After changing function f to $\phi =1/f$, we obtain

$${\phi }^{\prime }\left((1-p)t\right)\left(p+(1-p)\phi \left(t\right)\right)=\lambda \phi \left((1-p)t\right).$$

(18)

By putting $t=0$ and taking into account $\phi \left(0\right)=1$, we obtain ${\phi }^{\prime }\left(0\right)=\lambda$ (which is obvious). However, differentiating both parts of (18) with respect to t we obtain

$${\phi }^{″}\left((1-p)t\right)(1-p)\left(p+(1-p)\phi \left(t\right)\right)+(1-p){\phi }^{\prime }\left((1-p)t\right){\phi }^{\prime }\left(t\right)=\lambda (1-p){\phi }^{\prime }\left((1-p)t\right)$$

(19)

By putting $t=0$, we obtain ${\phi }^{″}\left(0\right)=0$. The induction shows that ${\phi }^{\left(m\right)}\left(0\right)=0$ for all $m=2,3,\dots$. This implies that $\phi \left(t\right)$ is a linear polynomial in t and, therefore, $f\left(t\right)$ is the Laplace transform of an exponential distribution. □

3. Conclusions

There are many other problems connected to the characterizations of the exponential distribution based on the properties of linear forms with random coefficients. Let us mention some questions regarding the identical distributions of quadratic forms, the constant of regression of quadratic statistics on linear forms with random coefficients, and the reconstruction of a distribution through the common distribution of a set of linear forms with random coefficients.

In ([14], p. 153–157), the authors provided a characterization of the Marshall–Olkin law based on the identical distribution property of a monomial and a linear form with a random matrix coefficient. It would be interesting to study the possibility of characterizing the Marshall–Olkin distribution by the independence of suitable statistics.

Some of the properties mentioned above will be subjects for future work.