On the Linear Combination of Exponential and Gamma Random Variables

Abstract

The exact distribution of the linear combination αX + βY is derived when X and Y are exponential and gamma random variables distributed independently of each other. A measure of entropy of the linear combination is investigated. We also provide computer programs for generating tabulations of the percentage points associated with the linear combination. The work is motivated by examples in automation, control, fuzzy sets, neurocomputing and other areas of computer science.

1 Introduction

For given random variables X and Y , the distribution of linear combinations of the form αX + βY is of interest in problems in automation, control, fuzzy sets, neurocomputing and other areas of computer science. Some examples are:

• In automatic control, one often encounters the problem of maximizing the expected sum of n variables, chosen from a sequence of N sequentially arriving i.i.d. scalar random variables, X₁, X₂, . . . , X_N. The objective is to devise a decision rule so as to maximize ${\displaystyle {\sum }_{i=1}^n X_{k_i}}$, where k_i ∈ {1, 2, . . . , N} is the index of the ith random variable selected. At time k, the random variable X_k is observed, and the decision to select the value or not must be taken online. This problem is known as the sequential screening problem and many decision problems can be formulated in this way (Pronzato [1]).
• The theory of congruence equations (see, for example, Cerruti [2]) has applications in computer science. There is a wide literature about congruence equations and the last twenty years has seen interesting formulas and functions derived: among these, expressions giving the number of solutions of linear congruences. Counting such solutions has relations with statistical problems like the distribution of the values taken by particular sums.
• In neurocomputing, linear combinations are used for combining multiple probabilistic classifiers on different feature sets. In order to achieve the improved classification performance, a generalized finite mixture model is proposed as a linear combination scheme and implemented based on radial basis function networks. In the linear combination scheme, soft competition on different feature sets is adopted as an automatic feature rank mechanism so that different feature sets can be always simultaneously used in an optimal way to determine linear combination weights (Chen and Chi [3]).
• Sums of random variables also have many applications in fuzzy sets and systems (see, for example, Boswell and Taylor [4], Williamson [5], Inoue [6], Jang and Kwon [7], and Feng [8,9]).

The distribution of αX + βY has been studied by several authors especially when X and Y are independent random variables and come from the same family. For instance, see Fisher [10] and Chapman [11] for Student’s t family, Christopeit and Helmes [12] for normal family, Davies [13] and Farebrother [14] for chi-squared family, Ali [15] for exponential family, Moschopoulos [16] and Provost [17] for gamma family, Dobson et al [18] for Poisson family, Pham-Gia and Turkkan [19] and Pham and Turkkan [20] for beta family, Kamgar-Parsi et al [21] and Albert [22] for uniform family, Hitezenko [23] and Hu and Lin [24] for Rayleigh family, and Witkovský [25] for inverted gamma family.

However, there is relatively little work of the above kind when X and Y belong to different families. In the applications mentioned above, it is quite possible that X and Y could arise from different but similar distributions. In this paper, we study the exact distribution of αX + βY when X and Y are independent random variables having the exponential and gamma distributions with pdfs

$$f_X\left(x\right)=\lambda \text{exp}(-\lambda x)$$

(1)

and

$$f_Y(y) = \frac{{\mu }^a{y}^{a-1}\exp (-\mu y)}{\Gamma \left(a\right)},$$

(2)

respectively, for x > 0, y > 0, λ > 0, µ > 0 and a > 0. We assume without loss of generality that α > 0.

The paper is organized as follows. In Section 2, we derive explicit expressions for the pdf and the cdf of αX + βY. A measure of entropy of the linear combination is investigated in Section 3. In Section 4, we provide computer programs for generating tabulations of the percentage points associated with the linear combination. We hope that these programs will be of use to the practitioners mentioned above.

The calculations of this paper involve several special functions, including the incomplete gamma function defined by

$$\gamma (a,x) = {\displaystyle {\int }_0^x{t}^{a-1}\exp (-t)}dt,$$

the complementary incomplete gamma function defined by

$$\Gamma (a,x) = {\displaystyle {\int }_x^{\infty }{t}^{a-1}\exp (-t)}dt,$$

and the error function defined by

$$\text{erfc}(x) = \frac{2}{\sqrt{\pi }}{\displaystyle {\int }_x^{\infty }\exp (-t^2)}dt.$$

The properties of the above special functions can be found in Prudnikov et al. [26] and Gradshteyn and Ryzhik [27].

2 PDF and CDF

Theorem 1 derives explicit expressions for the pdf and the cdf of αX + βY in terms of the incomplete gamma functions.

Theorem 1 

Suppose X and Y are distributed according to (1) and (2), respectively. The cdf of Z = αX + βY can be expressed as

$$F_Z(z) = \frac{1}{\Gamma \left(a\right)}\gamma (a,\frac{\mu z}{\beta })-\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}\exp (-\frac{\lambda z}{\alpha })\gamma (a,\frac{z(\mu \alpha -\lambda \beta )}{\alpha \beta })$$

(3)

for β > 0 and z > 0, as

$$F_Z(z) = \frac{1}{\Gamma \left(a\right)}\Gamma (a,\frac{\mu z}{\beta })-\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}\exp (-\frac{\lambda z}{\alpha })\Gamma (a,\frac{z(\mu \alpha -\lambda \beta )}{\alpha \beta })$$

(4)

for β < 0 and z < 0, and as

$$F_Z(z)=1-\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a}\exp (-\frac{\lambda z}{\alpha })$$

(5)

for β < 0 and z ≥ 0. The corresponding pdfs are:

$$f_Z(z) = \frac{\lambda {\left(\mu \alpha \right)}^a}{\alpha {(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}\exp (-\frac{\lambda z}{\alpha })\gamma (a,\frac{z(\mu \alpha -\lambda \beta )}{\alpha \beta })$$

(6)

for β > 0 and z > 0,

$$f_Z(z) = \frac{\lambda {\left(\mu \alpha \right)}^a}{\alpha {(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}\exp (-\frac{\lambda z}{\alpha })\Gamma (a,\frac{z(\mu \alpha -\lambda \beta )}{\alpha \beta })$$

(7)

for β < 0 and z < 0, and

$$f_Z(z)=\frac{\lambda {\left(\mu \alpha \right)}^a}{\alpha (\mu \alpha -\lambda \beta )}\exp (-\frac{\lambda z}{\alpha })$$

(8)

for β < 0 and z ≥ 0.

Proof: 

If β > 0 then the result follows by writing

$$\begin{array}{ll}\mathrm{Pr}(\alpha X+\beta Y\le z)\hfill & =\mathrm{Pr}(X\le \frac{z-\beta Y}{\alpha })\hfill \\ & ={\displaystyle {\int }_0^{z/\beta }Fx\left(\frac{z-\beta y}{\alpha }\right)f_Y(y)dy}\hfill \\ & =F_Y\left(\frac{z}{\beta }\right)-\frac{{\mu }^a\text{exp}(-\lambda z/\alpha )}{\Gamma \left(a\right)}{\displaystyle {\int }_0^{z/\beta }{y}^{a-1}\text{exp}(-\frac{\mu \alpha -\lambda \beta }{\alpha }y)dy}\hfill \\ & =\frac{1}{\Gamma \left(a\right)}\gamma (a,\frac{\mu z}{\beta })-\frac{{\mu }^a\exp (-\lambda z/\alpha )}{\Gamma \left(a\right)}\frac{{\alpha }^a}{{(\mu \alpha -\lambda \beta )}^{\alpha }}\gamma (a,\frac{z(\mu \alpha -\lambda \beta )}{\alpha \beta }),\hfill \end{array}$$

where the last step follows from the definition of the incomplete gamma function. The result in (4) can be established similarly by using the definition of the complementary incomplete gamma function. The result in (5) follows by setting z = 0 into to the two incomplete gamma function terms in (4). ■

The following corollaries provide the cdfs for the sum and the difference of the exponential and gamma random variables.

Corollary 1 

Suppose X and Y are distributed according to (1) and (2), respectively. Then, the cdf of Z = X + Y can be expressed as

$$F_Z(z) = \frac{1}{\Gamma \left(a\right)}\gamma (a,\mu z)-\frac{{\mu }^a}{{(\mu -\lambda )}^a\Gamma (a)}\exp (-\lambda z)\gamma (a,z(\mu -\lambda ))$$

for z > 0.

Corollary 2 

Suppose X and Y are distributed according to (1) and (2), respectively. Then, the cdf of Z = X − Y can be expressed as

$$F_Z(z) = \frac{1}{\Gamma (a)}\Gamma (a,\mu z)-\frac{{\mu }^a}{{(\mu +\lambda )}^a\Gamma (a)}\exp (-\lambda z)\Gamma (a,z(\mu +\lambda ))$$

for z < 0 and as

$$F_Z(z) = 1-\frac{{\mu }^a}{{(\mu +\lambda )}^a}\exp (-\lambda z)$$

for z ≥ 0.

Using special properties of the incomplete gamma functions, one can obtain simpler expressions for (3)–(4) when a takes integer or half integer values. This is illustrated in the corollaries below.

Corollary 3 

If a ≥ 1 is an integer then (3)–(4) can be reduced to the simpler forms

$$F_Z(z) = 1-\exp (-y){\displaystyle \sum _{k=0}^{a-1}\frac{y^k}{k!}-}\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a}\exp (-\frac{\lambda z}{\alpha })+\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a}\exp (-x-\frac{\lambda z}{\alpha }){\displaystyle \sum _{k=0}^{a-1}\frac{y^k}{k!}}$$

for β > 0 and z > 0, and

$$F_Z(z) = \exp (-y){\displaystyle \sum _{k=0}^{a-1}\frac{y^k}{k!}-}\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a}\exp (-x-\frac{\lambda z}{\alpha }){\displaystyle \sum _{k=0}^{a-1}\frac{x^k}{k!}}$$

for β < 0 and z < 0, where x = z(µα − λβ)/(αβ) and y = µz/β.

Corollary 4 

If a − 1/2 ≥ 0 is an integer then (3)–(4) can be reduced to the simpler forms

for β > 0 and z > 0, and

for β < 0 and z < 0, where x = z(µα − λβ)/(αβ) and y = µz/β.

Figure 1 below illustrates possible shapes of the pdfs (6)–(8) for selected values of α, β and a. The four curves in each plot correspond to selected values of a. As expected, the densities are unimodal and the effect of the parameters is evident.

3 Entropy

An entropy of a random variable is a measure of variation of the uncertainty. The simplest known entropy is the Shannon entropy (Shannon [28]) defined by

$$E[-\log f_Z(Z)] = -{\displaystyle \int \log f_Z\left(z\right)}{f}_Z(z)dz.$$

(9)

Consider calculating this when Z has the pdfs described in Theorem 1. If β > 0 then one can write where I denotes the integral

$$E[-\log f_Z(Z)]=-\log \left[\frac{\lambda {\left(\mu \alpha \right)}^a}{\alpha {(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}\right]+\frac{\lambda }{\alpha }E(z)-\frac{\lambda {\left(\mu \alpha \right)}^a}{\alpha {(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}I,$$

where I denotes the integral

$$I ={\displaystyle {\int }_0^{\infty }\log \gamma (a,\frac{z(\mu \alpha -\lambda \beta )}{\alpha \beta })\exp (-\frac{\lambda z}{\alpha })\gamma }(a,\frac{z(\mu \alpha -\lambda \beta )}{\alpha \beta })dz.$$

Unfortunately, this integral I cannot be reduced to a closed form even in the simplest case a = 1. Thus, one cannot obtain a closed form expression even for the simplest entropy measure when Z is distributed as in Theorem 1. Hence, we performed a numerical study to examine the behavior of (9) with respect to the parameters in Theorem 1. A program in R (Ihaka and Gentleman [29]) written to compute (9) is presented below.

cc<-lambda*((mu*alpha)**a)/(alpha*gamma(a)*(mu*alpha-lambda*beta)**a)
ff<-function (x)
{tt<-gamma(a)*pgamma(x*(mu*alpha-lambda*beta)/(alpha*beta),shape=a)
tt<-exp(-lambda*x/alpha)*tt*log(tt)
return(tt)}
ent<-1+lambda*beta*a/(alpha*mu)-log(cc)
ent<-ent-cc*integrate(ff,lower=0,upper=Inf)$value

Figure 2 below shows the variation of (9) for a range of values of α, β and a with λ = 1 and µ = 1. The effect of the parameters is evident: for fixed β, (9) is an increasing function of both α and a; for fixed α, (9) increases with respect to β but, with respect to a, it initially increases before decreasing.

One could also consider other more advanced measures of entropy such as the Rényi entropy defined by

$${\mathcal{J}}_R(\gamma )=\frac{1}{1-\gamma }\log \left\{{\displaystyle \int f_Z^{\gamma }\left(z\right)dz}\right\},$$

where γ > 0 and γ ≠ 1 (Rényi [30]). But, for the reasons mentioned above, one cannot obtain closed form expressions for these and the investigation will have to be performed numerically.

4 Percentiles

In this section, we provide two computer programs for generating tabulations of percentage points z_p associated with the cdf of Z = αX + βY. These percentiles are computed numerically by solving the equations

$$\begin{array}{c}\frac{1}{\Gamma \left(a\right)}\gamma (a,\frac{\mu z_p}{\beta })-\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}\exp (-\frac{\lambda z_p}{\alpha })\gamma (a,\frac{z_p(\mu \alpha -\lambda \beta )}{\alpha \beta })=p,\\ \frac{1}{\Gamma \left(a\right)}\Gamma (a,\frac{\mu z_p}{\beta })-\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a\Gamma \left(a\right)}\exp (-\frac{\lambda z_p}{\alpha })\Gamma (a,\frac{z_p(\mu \alpha -\lambda \beta )}{\alpha \beta })=p\end{array}$$

and

$$1-\frac{{\left(\mu \alpha \right)}^a}{{(\mu \alpha -\lambda \beta )}^a}\exp (-\frac{\lambda z_p}{\alpha })=p.$$

Evidently, this involves computation of the incomplete gamma and the complementary incomplete gamma functions and routines for this are widely available. We used the function GAMMA (·) in the algebraic manipulation package, MAPLE. The MAPLE programs below compute the percentiles z_p for p = 0.01, 0.05, 0.1, 0.90, 0.95, 0.99 for given values of α, β, λ, µ and a.

#this  program  gives  percentiles  when  beta  >  0
ff:=(1/GAMMA(a))*((mu*alpha)/(mu*alpha-lambda*beta))**a*exp(-lambda*z/alpha):
ff:=ff*(GAMMA(a)-GAMMA(a,z*(mu*alpha-lambda*beta)/(alpha*beta))):
ff:=1-GAMMA(a,mu*z/beta)/GAMMA(a)-ff:
p1:=fsolve(ff=0.01,z=0..1000):
p2:=fsolve(ff=0.05,z=0..1000):
p3:=fsolve(ff=0.1,z=0..1000):
p4:=fsolve(ff=0.90,z=0..1000):
p5:=fsolve(ff=0.95,z=0..1000):
p6:=fsolve(ff=0.99,z=0..1000):
print(p1,p2,p3,p4,p5,p6);
 
#this  program  gives  percentiles  when  beta  <  0
ff1:=(1/GAMMA(a))*((mu*alpha)/(mu*alpha-lambda*beta))**a:
ff1:=ff1*exp(-lambda*z/alpha):
ff1:=ff1*GAMMA(a,z*(mu*alpha-lambda*beta)/(alpha*beta)):
ff1:=GAMMA(a,mu*z/beta)/GAMMA(a)-ff1:
ff2:=1-((mu*alpha)/(mu*alpha-lambda*beta))**a*exp(-lambda*z/alpha):
bd:=1-((mu*alpha)/(mu*alpha-lambda*beta))**a:
if (bd>0.01) then p1:=fsolve(ff1=0.01,z=-1000..0): end if:
if (bd<=0.01) then p1:=fsolve(ff2=0.01,z=0..1000): end if:
if (bd>0.05) then p2:=fsolve(ff1=0.05,z=-1000..0): end if:
if (bd<=0.05) then p2:=fsolve(ff2=0.05,z=0..1000): end if:
if (bd>0.1) then p3:=fsolve(ff1=0.1,z=-1000..0): end if:
if (bd<=0.1) then p3:=fsolve(ff2=0.1,z=0..1000): end if:
if (bd>0.9) then p4:=fsolve(ff1=0.9,z=-1000..0): end if:
if (bd<=0.9) then p4:=fsolve(ff2=0.9,z=0..1000): end if:
if (bd>0.95) then p5:=fsolve(ff1=0.95,z=-1000..0): end if:
if (bd<=0.95) then p5:=fsolve(ff2=0.95,z=0..1000): end if:
if (bd>0.99) then p6:=fsolve(ff1=0.99,z=-1000..0): end if:
if (bd<=0.99) then p6:=fsolve(ff2=0.99,z=0..1000): end if:
print(p1,p2,p3,p4,p5,p6);

We hope these programs will be of use to the practitioners of the linear combination (see Section 1).