Cantelli’s Bounds for Generalized Tail Inequalities

Abstract

Let X be a centered random vector in a finite-dimensional real inner product space $\mathcal{E}$. For a subset C of the ambient vector space V of $\mathcal{E}$ and $x,y\in V$, write $x{⪯}_{C}y$ if $y-x\in C$. If C is a closed convex cone in $\mathcal{E}$, then ${⪯}_C$ is a preorder on V, whereas if C is a proper cone in $\mathcal{E}$, then ${⪯}_C$ is actually a partial order on V. In this paper, we give sharp Cantelli-type inequalities for generalized tail probabilities such as $\mathrm{Pr}\left\{X{⪰}_{C}b\right\}$ for $b\in V$. These inequalities are obtained by “scalarizing” $X{⪰}_{C}b$ via cone duality and then by minimizing the classical univariate Cantelli’s bound over the scalarized inequalities. Three diverse applications to random matrices, tails of linear images of random vectors, and network homophily are also given.

1. Introduction

Let Y be a random variable with finite mean $\mu$ and variance ${\sigma }^2$. Thus, the random variable $X=Y-\mu$ is centered and has the same variance as Y. For a positive real number b, the celebrated Cantelli inequality—also known as the one-sided Chebyshev’s inequality—reads as follows

$$\mathrm{Pr}\left\{X\ge b\right\}\le {\displaystyle \frac{{\sigma }^2}{b^2+{\sigma }^2}}.$$

(1)

Both Cantelli’s inequality and the classical Chebyshev’s inequality can be (and have been) extended in several ways [1,2] to a random vector $X={(X_1,\dots ,X_n)}^{\prime }$ in ${\mathbb{R}}^n$—here and throughout the rest of the paper, $u^{\prime }$ denotes the transpose of the column vector $u\in {\mathbb{R}}^n$. As shown in [1,2], there is a standard recipe that yields such extensions: Let X be a random vector supported by a subset S of ${\mathbb{R}}^n$ and let $\Sigma$ be the covariance matrix of X. Let T be a Borel subset of S and $f:S\to \mathbb{R}$ such that $f\left(x\right)\ge 0$ for all $x\in S$ and $f\left(x\right)\ge 1$ for all $x\in T$. Then, with ${\mathbf{1}}_T(·)$ denoting the indicator of the set T over S, one has $f\ge {\mathbf{1}}_T$ and

$$\mathbb{E}\left(f\left(X\right)\right)\ge \mathbb{E}\left(f\left(X\right){\mathbf{1}}_T\left(X\right)\right)\ge \mathbb{E}\left({\mathbf{1}}_T\left(X\right)\right)=\mathrm{Pr}\left\{X\in T\right\}.$$

This technique is essentially a “Markov inequality”. By taking f in the family $\left\{f_u\right|f_u:{\mathbb{R}}^n\to {\mathbb{R}}_{+}^n,u\in {\mathbb{R}}^n\}$, where $f_u\left(x\right)={\displaystyle \frac{{(u^{\prime }x+u^{\prime }\Sigma u)}^2}{1+u^{\prime }\Sigma u}}$, and minimizing for u under the constraint $f\ge {\mathbf{1}}_T$, Marshall and Olkin obtained the following strong and general result.

Theorem 1 

(Marshall and Olkin [1]). Let T be a closed convex set in ${\mathbb{R}}^n$ not containing the origin. If X is a centered random vector of ${\mathbb{R}}^n$ with a positive-definite covariance matrix Σ, then

$$\mathrm{Pr}\left\{X\in T\right\}\le \underset{\begin{array}{c}u\in {\mathbb{R}}^n\\ u^{\prime }x\ge 1,\forall x\in T\end{array}}{inf}{\displaystyle \frac{u^{\prime }\Sigma u}{1+u^{\prime }\Sigma u}}.$$

(2)

Furthermore, the inequality is sharp, in the sense that there exists a centered random vector $X_0$ whose support contains T and whose covariance matrix is Σ, such that the inequality is attained as an equality.

Note that Cantelli’s inequality (1) follows from inequality (2) after dividing the univariate random variable X by the positive threshold b and observing that the variance of $X/b$ is ${\sigma }^2/b^2$. The minimization problem on the right-hand side of (2) is solved by minimizing the quadratic form $u^{\prime }\Sigma u$ over the same set. This is a convex minimization problem that can be solved using the techniques described in [3]. As proved in [1], the infimum in (2) is attained. The function $f_{\widehat{u}}$ corresponding to the vector $\widehat{u}$, which attains the infimum in inequality (2), can be seen as a kind of envelope of a given shape (in this case quadratic) for the probability on the right-hand side. The same inequality can be interpreted in the following way: First, we linearly approximate T inside the probability. This approximation yields a family of linear inequalities, each of which is the tail of a scalar random variable. We then use Cantelli’s inequality (1) to bound each of these tails, and finally, we choose the tightest one. Let us describe this process for a non-empty arbitrary Borel subset T of ${\mathbb{R}}^n$: Let $\mathfrak{b}\left(T\right)=\{u\in {\mathbb{R}}^n|u^{\prime }x\ge 1,\forall x\in T\}$—note that $\mathfrak{b}\left(T\right)$ is always a closed convex set regardless of the argument T (see Section 2 for more details) and hence a Borel set—since, by definition, $T\subseteq \mathfrak{b}\left(\mathfrak{b}\right(T\left)\right)$ and $x\in \mathfrak{b}\left(\mathfrak{b}\left(T\right)\right)\iff u^{\prime }x\ge 1,\forall u\in \mathfrak{b}\left(T\right)$, it follows that

$$\mathrm{Pr}\left\{X\in T\right\}\le \mathrm{Pr}\left\{X\in \mathfrak{b}\left(\mathfrak{b}\right(T\left)\right)\right\}=\mathrm{Pr}\left\{u^{\prime }X\ge 1,\forall u\in \mathfrak{b}\left(T\right)\right\},$$

where X is a centered random vector with a positive-definite covariance matrix $\Sigma$. Here, for all $u\in \mathfrak{b}\left(T\right)$, the random variable $u^{\prime }X$ is a centered random variable with variance $u^{\prime }\Sigma u$. Thus, by Cantelli’s inequality, for all $u\in \mathfrak{b}\left(T\right)$, it holds that

$$\mathrm{Pr}\left\{X\in T\right\}\le {\displaystyle \frac{u^{\prime }\Sigma u}{1+u^{\prime }\Sigma u}}.$$

Thus, if T is a non-empty convex set, after taking the infimum over $\mathfrak{b}\left(T\right)$, we recover (2) from another perspective. It is easy to see that the same result holds in any finite-dimensional Euclidean space $\mathcal{E}$. If T is of the form $b+C$, where C is a convex cone in $\mathcal{E}$, then we provide a specialized Cantelli bound that is sharp. Furthermore, if C has a non-empty interior, then C induces a preorder ${⪰}_C$ on the ambient space of $\mathcal{E}$ such that the event $(X\in T)$ can be written as $\left(X{⪰}_{C}b\right)$ and can be interpreted as a generalized tail inequality (we recover classical tail inequalities when C is the non-negative orthant). Thus, Cantelli’s inequality naturally extends to generalized tail inequalities in finite-dimensional Euclidean spaces. While such an extension is not really genuine since, in the finite-dimensional case, it can be derived by standard arguments from the special case of the standard real n-dimensional Euclidean space (see Lemma 7), on the other hand, it allows a tight sharpening of the general case (Corollary 1) and also allows to cast seemingly more complicated events into the framework of tail inequalities, as they occur, for example, in random matrix theory. Consider the case where X is a random matrix sampled from a symmetric real ensemble (see Section 4.1 for precise definitions). A fundamental problem in this context is to understand, at least asymptotically, when the order of the matrix goes to ∞, the probability that the smallest eigenvalue of X is positive. This problem is completely and precisely solved for Gaussian matrices (see [4,5,6]): the probability of sampling a positive-definite matrix from a symmetric Gaussian ensemble goes to zero exponentially fast. In this paper, using Corollary 1 in Section 3, we prove that the same result holds more generally for the Wigner matrices of the form ${\displaystyle \frac{M+M^{\prime }}{2}}$, where M is a random matrix whose entries are i.i.d. centered random variables, although the rate we can provide is much weaker. The result follows by looking at the random matrix X as a random vector in the Euclidean space $\mathcal{E}=(S^n,{\langle ·,·\rangle }_F)$, where $S^n$ is the real vector space of real symmetric matrices and ${\langle ·,·\rangle }_F$ is the Frobenius inner product. If $S_{+}^n$ denotes the cone of positive semi-definite matrices lying in $\mathcal{E}$ and ⪰ is the partial order induced by $S_{+}^n$, then, for a real number $ϵ$, with I being the identity matrix of order n, the generalized tail $\left(X{⪰}_{S_{+}^n}ϵI\right)$ is the event that occurs when the least eigenvalue of X is at least $ϵ$. Another similar motivation for studying generalized tails comes from the need to compute the probability of the feasibility of systems of linear inequalities whose unknowns are real random variables and the coefficients of the system are deterministic (see Section 4.2). This problem can be cast in the context of concentration and tail inequalities for linear, possibly random images of random vectors [7,8]: Ref. [8] studies the related problem of determining general concentration inequalities, while [7] actually investigates the more general case of tails of random linear images of random vectors, which, in our interpretation, corresponds to the case where both the coefficients and the unknowns of the system are random variables. In this paper, we give a Cantelli-type bound on the feasibility of a system of linear inequalities with random unknowns and show that the “ordinary tails” of the linear image $f\left(X\right)$ of a random vector X under a surjective map f are just generalized tails for the random vector X itself taken with respect to a polyhedral cone. Equivalently, by pursuing the interpretation of tails of linear images as solution sets of a linear system of inequalities in the random vector X, we show that if the system has a right-invertible coefficient matrix, then its solution set is a generalized tail for X taken with respect to a polyhedral cone: the rows of the coefficient matrix are the normal vectors of the proper maximal faces of the cone.

It is clear from the discussion above that tail inequalities for random vectors depend crucially on their covariance functions $\Sigma$. When $\Sigma$ has some additional structure, for special choices of the threshold b, Cantelli’s inequality for generalized tails reads as a very simple expression in terms of certain norms of b. For instance (see Section 3 for details), for $b\ge 0$, if ${\Sigma }^{-1}$ is a non-negative matrix, then

$$\mathrm{Pr}\left\{X\ge b\right\}\le {(1+\parallel b\parallel }_M^2{)}^{-1},$$

where ${\parallel b\parallel }_M=\sqrt{b^{\prime }{\Sigma }^{-1}b}$ is the Mahalanobis norm of b. Note that inequality (1) specializes to the inequality above after dividing the numerator and denominator by ${\sigma }^2$. The assumption ${\Sigma }^{-1}\ge 0$ is only seemingly artificial. It is often satisfied by finitely supported random vectors that occur in network science. In Section 4.3, we will give one such important case. In light of Theorem 7, which rests on observations from [9], having easily computable bounds on tail probabilities, like the one above, is generally the best we can hope for, even if we have no control over the quality of the approximation.

The rest of the paper is organized as follows: In Section 2, we develop the machinery to state and prove the main results provided in Section 3, while in Section 4, we apply our results to the probability of sampling a positive-definite Wigner matrix, to the probability of feasibility of a system of linear inequalities, and to testing homophily in networks.

2. Preparatory Results

Finite-dimensional Euclidean spaces, namely, finite-dimensional real vector spaces V equipped with an inner product $\langle ·,·\rangle$, are denoted by calligraphic letters such as $\mathcal{E}$. The symbol ${\mathcal{R}}^n$ stands for the Euclidean space $({\mathbb{R}}^n,\langle ·,·\rangle )$, where $\langle ·,·\rangle$ is the standard dot product in ${\mathbb{R}}^n$. In the following, when we talk about Euclidean spaces, we mean finite-dimensional Euclidean spaces over reals.

When dealing with random vectors, the Borel sets we are interested in have the form $b+C$, where $b\in V$ and $C\subseteq V$ is a non-empty Borel set, often a cone. A cone in $\mathcal{E}$ is a subset C of V that is closed under taking positive scalar multiples, i.e., $\lambda C\subseteq C$ for every $\lambda >0$, while a convex cone C is a cone that is closed under taking sum, i.e., $C+C\subseteq C$. The empty set, the set consisting only of the zero vector of V, and V itself are the trivial cones. In the following, when we speak of cones, we mean non-trivial cones. Crucial to the definition of generalized inequality is the following notion of duality of subsets of Euclidean spaces. First, identify the algebraic dual $V^{*}$ of V with V by the inner product in $\mathcal{E}$ via the isomorphism $V\ni v\mapsto \langle v,·\rangle \in V^{*}$. Let C be a non-empty subset of V. The dual $C^{*}$ of C in $\mathcal{E}$ is the set

$$C^{*}=\{u\in V|\langle u,v\rangle \ge 0,\forall v\in C\}.$$

Write $C^{**}$ for the double dual of C, namely, ${\left(C^{*}\right)}^{*}$. The following facts, the first three of which are simple consequences of the definition, are known about the dual of a non-empty set C (see [10,11]).

Lemma 1. 

Let $C\subseteq V$. Then,

(a) 

$C^{*}$ is always a closed convex cone;

(b) 

$C_1\subseteq C_2\Rightarrow C_1^{*}\supseteq C_2^{*}$, i.e., duality is inclusion reversing;

(c) 

$C\subseteq C^{**}$;

(d) 

$C=C^{**}$ if and only if C is a closed convex cone.

Proof. 

See [10,11].    □

The last property in the lemma implies that $C^{*}=C^{***}$ because $C^{*}$ is a closed convex cone by (a). A cone C is proper whenever C is a closed convex cone that is also pointed, i.e., $C\cap -C=\left\{0\right\}$, where 0 is the zero vector of V, and has non-empty interior. A cone C is self-dual in $\mathcal{E}$ if $C=C^{*}$.

For a random vector $X={(X_1,\dots ,X_n)}^{\prime }\in {\mathbb{R}}^n$ and $b={(b_1,\dots ,b_n)}^{\prime }\in {\mathbb{R}}^n$, the event $(X\ge b)$ is said to be a tail of X. Such an event reads as $(X_1\ge b_1,\dots ,X_n\ge b_n)$ and is the same event as $X-b\in {\mathbb{R}}_{+}^n$. The non-negative orthant ${\mathbb{R}}_{+}^n$ is a self-dual cone in ${\mathcal{R}}^n$. Clearly, $X-b$ is non-negative if and only if $u^{\prime }(X-b)$ is non-negative for all vectors $u\in {\mathbb{R}}_{+}^n$, i.e., for all u in the dual cone of ${\mathbb{R}}_{+}^n$. This fact can be generalized as follows. Let the ambient space of $\mathcal{E}$ be V and let C be a non-empty subset of V. For $x,y\in V$, write $y{⪰}_{C}x$ if $y-x\in C$. If C is a convex cone, then ${⪰}_C$ is a pre-order on V, while if C is a proper cone, then ${⪰}_C$ is a partial order on V. In any case, even when C is arbitrary, by duality, one has

$$y{⪰}_{C}x⟹y{⪰}_{C^{**}}x⟺\langle u,y-x\rangle \ge 0,\forall u\in C^{*}$$

(3)

which reduces to

$$y{⪰}_{C}x⟺\langle u,y-x\rangle \ge 0,\forall u\in C^{*}$$

when C is a closed convex cone. Generalized inequalities, and hence generalized tails, are well behaved with respect to linear transformations of random vectors because cones are preserved by such maps. In fact, if the latter are invertible, closedness is preserved as well.

We also need a lesser-known duality device, which we borrow from the theory of blocking pairs of polyhedra [12]. Let V be the ambient space of $\mathcal{E}$. For $T\subseteq V$, the blocker of T in $\mathcal{E}$ is the set

$$\mathfrak{b}\left(T\right)=\{u\in V|\langle u,x\rangle \ge 1,\forall x\in T\}.$$

Analogous to the dual of T, the blocker of T has the following properties, the first three of which are straightforward.

Lemma 2. 

In the Euclidean space $\mathcal{E}=(V,\langle ·,·\rangle )$, let $T\subseteq V$. It holds that

(i) 

if $T\ne \varnothing$, then $\mathfrak{b}\left(T\right)$ is a non-empty closed convex set;

(ii) 

if $T\subseteq T^{\prime }$, then $\mathfrak{b}\left(T\right)\supseteq \mathfrak{b}\left(T^{\prime }\right)$;

(iii) 

$T\subseteq \mathfrak{b}\left(\mathfrak{b}\right(T\left)\right)$;

(iv) 

if $T=b+C$ where $b\in V$ and C is such that $\varnothing \ne C\subseteq V$, then

$$\mathfrak{b}(b+C)\supseteq \mathfrak{b}(b+C^{**})\supseteq C^{*}\cap \{u\in V|\langle u,b\rangle \ge 1\};$$

(4)

moreover, if C is a nontrivial cone, then

$$C^{*}\cap \{u\in V|\langle u,b\rangle >0\}\supseteq \mathfrak{b}(b+C)=C^{*}\cap \{u\in V|\langle u,b\rangle \ge 1\}.$$

(5)

Proof. 

If $T\ne \varnothing$, then $x/\sqrt{\langle x,x\rangle }\in \mathfrak{b}\left(T\right)$ for all $x\in T$. Hence, if T is non-empty, then so is $\mathfrak{b}\left(T\right)$. Moreover, since $\mathfrak{b}\left(T\right)$ is the intersection of closed half-spaces, then $\mathfrak{b}\left(T\right)$ is a closed convex set. This establishes (i). Statements (ii) and (iii) are straightforward. Let us prove (iv). To prove (4), first observe that if $u\in C^{*}\cap \{u\in V|\langle u,b\rangle \ge 1\}$, then $\langle u,b+y\rangle \ge 1,\forall y\in C^{**}$. Hence, $u\in \mathfrak{b}(b+C^{**})$ and $u\in \mathfrak{b}(b+C)$ because $\mathfrak{b}(b+C)\supseteq \mathfrak{b}(b+C^{**})$ by (ii). Let us prove (5). Since $C^{*}\cap \{u\in V|\langle u,b\rangle \ge 1\}\subseteq C^{*}\cap \{u\in V|\langle u,b\rangle >0\}$, it follows that to prove (5) it suffices to prove that $\mathfrak{b}(b+C)=C^{*}\cap \{u\in V|\langle u,b\rangle \ge 1\}$, which follows by $\mathfrak{b}(b+C)\subseteq C^{*}\cap \{u\in V|\langle u,b\rangle \ge 1\}$ after (4). Let us prove the latter inclusion. Note that if $u\in \mathfrak{b}(b+C)$ and C is a cone, then necessarily $\langle u,y\rangle \ge 0$ for all $y\in C$ for, if not, there exists $y_0\in C$ such that $\langle u,y_0\rangle <0$ and $b+\lambda y_0\in T$ for all $\lambda >0$. Hence, $\langle u,b+\lambda y_0\rangle <1$ for sufficiently large $\lambda$, which contradicts $u\in \mathfrak{b}(b+C)$. We conclude that $\langle u,y\rangle \ge 0$ for all $y\in C$, and thus, $\mathfrak{b}(b+C)\subseteq C^{*}$. By the same reasoning, it holds that $u\in \mathfrak{b}(b+C)\Rightarrow \langle u,b\rangle \ge 1$. To see this, assume by contradiction that $\langle u_0,b\rangle <1$ for some $u_0\in \mathfrak{b}(b+C)$. Since $u_0\in \mathfrak{b}(b+C)$, there exists $y_0\in C$, $y_0\ne 0$, such that $\langle u_0,b+\lambda y_0\rangle \ge 1$ for all $\lambda >0$. However, the latter inequality cannot be satisfied for a small enough $\lambda$. We conclude that the desired inclusion is true.    □

Remark 1. 

Usually the blocker of a polyhedron T of the form $P+{\mathbb{R}}_{+}^n$ is defined as the set $B\left(T\right)=\{u\in {\mathbb{R}}_{+}^n|u^{\prime }x\ge 1,\forall x\in T\}$. Hence, $B\left(T\right)=\mathfrak{b}\left(T\right)\cap {\mathbb{R}}_{+}^n$ in ${\mathcal{R}}^n$. However, the two definitions coincide in this case because, reasoning as in the proof of (5), $\mathfrak{b}\left(T\right)\subseteq {\left({\mathbb{R}}_{+}^n\right)}^{*}={\mathbb{R}}_{+}^n$. Therefore, $\mathfrak{b}\left(T\right)=B\left(T\right)$.

In the Euclidean space $\mathcal{E}=(V,\langle ·,·\rangle )$, for a set C and a vector b in V the set

$$\mathfrak{a}(b,C)=C^{*}\cap \{u\in V|\langle u,b\rangle >0\}$$

plays an essential role in the following intermediate results.

Lemma 3. 

Let C be a set in the Euclidean space $\mathcal{E}=(V,\langle ·,·\rangle )$. One has $\mathfrak{a}(b,C)\ne \varnothing$ if and only if $0\notin b+C^{**}$. Moreover, if C is a cone, then one has $\mathfrak{a}(b,C)\ne \varnothing$ if and only if $\mathfrak{b}(b+C)\ne \varnothing$.

Proof. 

To prove the first assertion, note that

$$\mathfrak{a}(b,C)=\varnothing ⟺\langle u,b\rangle \le 0\forall u\in C^{*}⟺-b\in C^{**}⟺0\in b+C^{**}.$$

For the second assertion observe that $\mathfrak{b}(b+C)\ne \varnothing \Rightarrow \mathfrak{a}(b,C)\ne \varnothing$ by (iv) in Lemma 2. Conversely, if $u\in \mathfrak{a}(b,C)$, then ${\displaystyle \frac{u}{\langle u,b\rangle }}\in \mathfrak{b}(b+C)$ by definition.    □

Note that the condition $0\notin b+C^{**}$ in Lemma 3 can be written as $b\in V\setminus -C^{**}$.

Lemma 4. 

In the Euclidean space $\mathcal{E}=(V,\langle ·,·\rangle )$, let C be a non-trivial cone, and $b\in V\setminus -C^{**}$. If $f,g:V\to \mathbb{R}$ are defined by

$$f\left(u\right)={\displaystyle \frac{q\left(u\right)}{{\langle u,b\rangle }^2+q\left(u\right)}}\mathit{and}g\left(u\right)={\displaystyle \frac{q\left(u\right)}{1+q\left(u\right)}},$$

where $q:V\to \mathbb{R}$ is a positive-definite quadratic form, then

$$\underset{\mathfrak{a}(b,C)}{inf}f\left(u\right)=\underset{\mathfrak{b}(b+C)}{inf}g\left(u\right).$$

Proof. 

The assumption on b guarantees that $\mathfrak{a}(b,C)$ and $\mathfrak{b}(b+C)$ are both non-empty by Lemma 3. Since q is a quadratic form, it is homogeneous of degree 2. Hence, f is homogeneous of degree 0, namely, $f\left(\lambda u\right)=f\left(u\right)$. Observe that $f\left(u\right)\le 1$ for all $u\in V$ and that $f\left(u\right)=1$ over the hyperplane $\{u\in V|u^{\prime }b=0\}$. Thus, by homogeneity, for every $u\in C^{*}$, f is constant on the rays $\lambda u$, $\lambda >0$. In particular,

$$f\left(u\right)=f\left({\displaystyle \frac{u}{\langle u,b\rangle }}\right)=g\left({\displaystyle \frac{u}{\langle u,b\rangle }}\right),\forall u\in C^{*}\cap \{u\in V|\langle u,b\rangle >0\}.$$

(6)

Since, by (5), it holds that $\mathfrak{b}(b+C)=C^{*}\cap \{u\in V|\langle u,b\rangle \ge 1\}$, it follows that

$$f\left(u\right)={\displaystyle \frac{q\left(u\right)}{{\langle u,b\rangle }^2+q\left(u\right)}}\le {\displaystyle \frac{q\left(u\right)}{1+q\left(u\right)}}=g\left(u\right)\forall u\in \mathfrak{b}(b+C).$$

(7)

Let $H=\{u\in V|\langle u,b\rangle =1\}$ and observe that

$$\begin{array}{cc}\hfill \underset{\mathfrak{a}(b,C)}{inf}f\left(u\right)& =\underset{\mathfrak{a}(b,C)\cap H}{inf}g\left(u\right)\left(\mathrm{by}(6\right))\hfill \\ & \ge \underset{\mathfrak{b}(b+C)}{inf}g\left(u\right)\left(\mathrm{because}\mathfrak{a}(b,C)\cap H\subseteq \mathfrak{b}(b+C)\mathrm{by}\right(5\left)\right)\hfill \\ & \ge \underset{\mathfrak{b}(b+C)}{inf}f\left(u\right)\left(\mathrm{by}\right(7\left)\right)\hfill \\ & \ge \underset{\mathfrak{a}(b,C)}{inf}f\left(u\right)\left(\mathrm{because}\mathfrak{a}(b,C)\supseteq \mathfrak{b}(b+C)\mathrm{by}\right(5\left)\right)\hfill \end{array}$$

Therefore, equality must hold throughout, yielding the desired equality.    □

Recall that given a self-adjoint positive-definite endomorphism of V in $\mathcal{E}=(V,\langle ·,·\rangle )$, the bilinear form $\langle u,A\left(v\right)\rangle$ induces a norm p on V by $p\left(v\right)=\sqrt{\langle v,A\left(v\right)\rangle }$. The dual norm of p (see [13], Section 5.4) is the norm $p^{*}$ such that

$$p^{*}\left(v\right)=\underset{u\ne 0}{sup}{\displaystyle \frac{|\langle u,v\rangle |}{\sqrt{\langle u,A\left(u\right)\rangle }}}.$$

Lemma 5. 

Let A be a self-adjoint positive-definite endomorphism of V in $\mathcal{E}=(V,\langle ·,·\rangle )$. If p is the norm on V defined by $p\left(v\right)=\sqrt{\langle v,A\left(v\right)\rangle }$, then $p^{*}\left(v\right)=\sqrt{\langle v,A^{-1}\left(v\right)\rangle }$.

Proof. 

Although this is actually a “folklore” fact, we were unable to locate a reference. Therefore, we provide a proof here.

Since A is a self-adjoint positive-definite endomorphism, $A=B^2$ for some self-adjoint positive-definite endomorphism B (B is a square-root of A). Hence, by the Cauchy–Schwarz inequality and because B is self-adjoint,

$$\begin{array}{cc}\hfill \left|\langle u,v\rangle \right|=|\langle u,B{B}^{-1}\left(v\right)\rangle |& =|\langle B\left(u\right),B^{-1}\left(v\right)\rangle |\hfill \\ & \le \sqrt{\langle B\left(u\right),B\left(u\right)\rangle }\sqrt{\langle B^{-1}\left(v\right),B^{-1}\left(v\right)\rangle }\hfill \\ & =\sqrt{\langle u,A\left(u\right)\rangle }\sqrt{\langle v,A^{-1}\left(v\right)\rangle }.\hfill \end{array}$$

Therefore, $p^{*}\left(v\right)\le \sqrt{\langle v,A^{-1}\left(v\right)\rangle }$ with equality for $v=0$. On the other hand, if $v\ne 0$, then the vector $\overline{v}$ defined by

$$\overline{v}={\displaystyle \frac{A^{-1}\left(v\right)}{\sqrt{\langle v,A^{-1}\left(v\right)\rangle }}}$$

is such that

$$p^{*}\left(v\right)\ge {\displaystyle \frac{|\langle \overline{v},v\rangle |}{\sqrt{\langle \overline{v},A\left(\overline{v}\right)\rangle }}}=\sqrt{\langle v,A^{-1}\left(v\right)\rangle }.$$

Hence, $p^{*}$ has the stated expression.    □

Lemma 6. 

In the Euclidean space $\mathcal{E}=(V,\langle ·,·\rangle )$, let C be a closed convex cone, $b\in V\setminus -C$, and p the norm $p\left(v\right)=\sqrt{\langle v,A\left(v\right)\rangle }$ on V, where A is a self-adjoint positive-definite endomorphism of V. If $b\in A\left(C^{*}\right)$, then, with $q=p^2$, in the notation of Lemma 4,

$$\underset{\mathfrak{a}(b,C)}{inf}f\left(u\right)={\displaystyle \frac{1}{{\left(p^{*}\left(b\right)\right)}^2+1}}$$

where $p^{*}$ is the dual norm of p, namely,

$$p^{*}\left(b\right)=\underset{u\ne 0}{sup}{\displaystyle \frac{\langle u,b\rangle }{p\left(u\right)}}=\sqrt{\langle b,A^{-1}\left(b\right)\rangle }.$$

Proof. 

Since C is a closed convex cone, then $C=C^{**}$ by (d). Hence, the assumption on b guarantees that $\mathfrak{a}(b,C)$ is non-empty by Lemma 3. Therefore, we can divide the numerator and the denominator of $f\left(u\right)$ by $q\left(v\right)$. This yields

$$\underset{\mathfrak{a}(b,C)}{inf}f\left(u\right)={\left(\underset{\mathfrak{a}(b,C)}{sup}{\left\{{\displaystyle \frac{|\langle u,b\rangle |}{p\left(u\right)}}\right\}}^2+1\right)}^{-1}.$$

Here, if $b\in A\left(C^{*}\right)$, then $A^{-1}\left(b\right)\in C^{*}$. Hence, $\widehat{b}={\displaystyle \frac{A^{-1}\left(b\right)}{\sqrt{\langle b,A^{-1}\left(b\right)\rangle }}}$ belongs to $\mathfrak{a}(C,b)$ because $\widehat{b}\in C^{*}$, with $\widehat{b}$ being a positive scalar multiple of $A^{-1}\left(b\right)$, and $\langle \widehat{b},b\rangle =\sqrt{\langle b,A^{-1}\left(b\right)\rangle }>0$. By Lemma 5, after plugging $\widehat{b}$ into ${\displaystyle \frac{\langle u,b\rangle }{p\left(u\right)}}$, we conclude that the supremum $p^{*}\left(b\right)$ of ${\displaystyle \frac{|\langle u,b\rangle |}{p\left(u\right)}}$ is attained over $\mathfrak{a}(b,C)$, and this concludes the proof.    □

3. Results

Let $(\Omega ,\mathcal{F},\mathbb{P})$ be a probability space, where $\Omega$ is a set, $\mathcal{F}$ is a $\sigma$-algebra on $\Omega$, and $\mathbb{P}$ is a probability measure on $\mathcal{F}$. Also, let $\mathcal{E}=(V,\langle ·,·\rangle )$ be a Euclidean space and $\mathcal{B}$ the smallest $\sigma$-algebra containing all the open balls of V taken with respect to the norm induced by $\langle ·,·\rangle$—this $\sigma$-algebra does not depend on the particular inner product chosen on V. A random vector X in $\mathcal{E}$ is a $(\mathcal{F},\mathcal{B})$-measurable map $X:\Omega \to V$. The algebra $\mathcal{B}$ is the algebra of Borel sets of $\mathcal{E}$. To prove our results, we first need the following lemma.

Lemma 7. 

Let X be a centered random vector in $\mathcal{E}=(V,\langle ·,·\rangle )$ with a positive-definite covariance function Σ. Let T be a non-empty Borel subset of V and $b\in V$, $b\ne 0$. Then, for all $u\in \mathfrak{b}\left(T\right)$, it holds that

$$\mathrm{Pr}\left\{X\in T\right\}\le {\displaystyle \frac{\langle u,\Sigma \left(u\right)\rangle }{1+\langle u,\Sigma \left(u\right)\rangle }}.$$

Therefore, if $\mathfrak{b}\left(T\right)$ is non-empty, then

$$\mathrm{Pr}\left\{X\in T\right\}\le \underset{u\in \mathfrak{b}\left(T\right)}{inf}{\displaystyle \frac{\langle u,\Sigma \left(u\right)\rangle }{1+\langle u,\Sigma \left(u\right)\rangle }}.$$

If T is a closed convex set such that $0\notin T$, then the above bound is sharp.

Proof. 

The second inequality follows from the first provided that $\mathfrak{b}\left(T\right)$ is non-empty. The proof of the first inequality is formally identical to the proof of the same inequality in ${\mathcal{R}}^n$ given in Section 1 and is a direct consequence of (ii) and Cantelli’s inequality (1) after noticing that, for a random vector X in $\mathcal{E}$, the variance of the random variable $\langle u,X\rangle$ is $\langle u,\Sigma u\rangle$. It remains to be shown that if T is closed and convex in $\mathcal{E}$ and $0\notin T$, then the bound is sharp. We deduce this result from Theorem 1 after reducing the general case to ${\mathcal{R}}^n$ by the following argument. Every Euclidean space is a topological vector space with respect to the standard topology induced by the inner product (recall that, in our terminology, Euclidean spaces are finite-dimensional and, hence, Hilbert spaces). Euclidean spaces of the same dimension are pairwise homeomorphic, and all are homeomorphic to ${\mathcal{R}}^n$ under the coordinate map isomorphism f (with respect to a fixed orthonormal basis). Thus, if X is a centered random vector in $\mathcal{E}$, then $f\left(X\right)$ is a centered random vector in ${\mathcal{R}}^n$, and conversely. Moreover, if the covariance function $\Sigma$ of X is positive-definite, then the covariance matrix $\tilde{\Sigma }$ of $f\left(X\right)$ is positive-definite, and conversely. Furthermore, T is a convex closed set in $\mathcal{E}$ if and only if $f\left(T\right)$ is a closed set in ${\mathcal{R}}^n$ (with the standard Euclidean topology): linear images of convex sets are convex, and homeomorphic linear images of closed convex sets are closed and convex. Finally, since f maps the zero vector $0_V$ of $\mathcal{E}$ to the zero vector 0 of ${\mathcal{R}}^n$, it follows that $0_V\notin T$ if and only if $0\notin f\left(T\right)$. Here,

$$\mathrm{Pr}\left\{X\in T\right\}=\mathrm{Pr}\left\{f\left(X\right)\in f\left(T\right)\right\}.$$

Therefore, if ${\tilde{X}}_0$ is a random vector in ${\mathcal{R}}^n$ with a positive-definite matrix $\tilde{\Sigma }$ that attains the bound in Theorem 1 as equality, which exists because the hypotheses on $f\left(T\right)$ are satisfied, then $f^{-1}\left({\tilde{X}}_0\right)$ is a random vector in $\mathcal{E}$ that attains the bound in this lemma.    □

Theorem 2. 

Let X be a centered random vector of $\mathcal{E}=(V,\langle ·,·\rangle )$ with positive-definite covariance Σ. Let C be a non-empty Borel subset of V and $b\in V$, $b\ne 0$. Then, for all $u\in C^{*}$ such that $\langle u,b\rangle >0$, it holds that

$$\mathrm{Pr}\left\{X{⪰}_{C}b\right\}\le {\displaystyle \frac{\langle u,\Sigma \left(u\right)\rangle }{{\langle u,b\rangle }^2+\langle u,\Sigma \left(u\right)\rangle }}.$$

If C is a closed convex cone in $\mathcal{E}$ and $b\in V\setminus -C$, then $\mathfrak{a}(b,C)$ is non-empty and

$$\mathrm{Pr}\left\{X{⪰}_{C}b\right\}\le \underset{u\in \mathfrak{a}(b,C)}{inf}{\displaystyle \frac{\langle u,\Sigma \left(u\right)\rangle }{{\langle u,b\rangle }^2+\langle u,\Sigma \left(u\right)\rangle }}=\underset{u\in \mathfrak{b}(b+C)}{inf}{\displaystyle \frac{\langle u,\Sigma \left(u\right)\rangle }{1+\langle u,\Sigma \left(u\right)\rangle }}.$$

Moreover, the above bound is sharp.

Proof. 

Since $\mathrm{Pr}\left\{X{⪰}_{C}b\right\}\le \mathrm{Pr}\left\{X{⪰}_{C^{**}}b\right\}$, the first inequality follows from (3) (with X instead of y and b instead of x) by applying Cantelli’s inequality (1) to the random variable $\langle u,X\rangle$. If C is a closed convex cone and $b\in V\setminus -C$, then $\mathfrak{a}(b,C)$ is non-empty because the hypotheses of Lemma 3 are satisfied (recall that since C is a closed convex cone, one has $C=C^{**}$ by (c) in Lemma 1). Moreover, still by Lemma 3, $\mathfrak{b}(b+C)$ is non-empty as well. Hence, the second inequality follows from the first by Lemma 4 with $q\left(u\right)=\langle u,\Sigma \left(u\right)\rangle$. It remains to be proven that the bound is sharp. Let $T=b+C$. Since C is closed and convex, then so is T. Moreover, the hypotheses on b and C implies $0\notin T$. Therefore, Lemma 7 applies, and we conclude that the bound is sharp.    □

Although the previous theorem deals with the special case of $T=b+C$ of Lemma 7, which, in turn, up to technicalities, follows by Theorems 1 and 2 provides a useful sharpening of the extended Cantelli’s inequality given in Theorem 1, which is further exploited in the next corollary.

Corollary 1. 

Let X be a centered random vector of $\mathcal{E}=(V,\langle ·,·\rangle )$ with positive-definite covariance Σ. Let C be a closed convex cone in $\mathcal{E}$. If $b\in \Sigma \left(C^{*}\right)\setminus -C$, where $\Sigma \left(C^{*}\right)$ denotes the linear image of $C^{*}$ under Σ, then

$$\mathrm{Pr}\left\{X{⪰}_{C}b\right\}\le {\displaystyle \frac{1}{1+{\parallel b\parallel }_{{\Sigma }^{-1}}^2}},$$

(8)

where ${\parallel b\parallel }_{{\Sigma }^{-1}}=\langle b,{\Sigma }^{-1}\left(b\right)\rangle$. Moreover, the bound is sharp.

Proof. 

Directly from Lemmas 4 and 6, with $p^{*}=\sqrt{\langle b,{\Sigma }^{-1}\left(b\right)\rangle }$.    □

The previous corollary can be further (and straightforwardly) specialized in ${\mathcal{R}}^n$ if X has a monotone covariance matrix. Recall that an invertible real matrix A is monotone if $A^{-1}$ has nonnegative entries (see [14], Chapter 6). Random vectors whose covariance matrix has this property are common in Network Science. See Section 4.3 for such an example.

Corollary 2. 

Let X be a centered random vector in ${\mathcal{R}}^n$ with a positive monotone covariance matrix Σ. Let C be a closed convex cone in ${\mathcal{R}}^n$ contained in the orthant ${\mathbb{R}}_{+}^n$. If $b\in {\mathbb{R}}_{+}^n$, then the hypotheses of Corollary 1 are satisfied, and hence, (5) holds.

Proof. 

We only need to show that the hypotheses of the present corollary guarantee that the hypotheses of Corollary 1 are met, namely,

$$\left\{\begin{array}{c}\hfill C={\mathbb{R}}_{+}^n\\ \hfill b\ge 0\\ \hfill {\Sigma }^{-1}\ge 0\end{array}\right.⟹b\in \Sigma \left(C^{*}\right)\setminus -C.$$

Clearly, $\left({\Sigma }^{-1}\ge 0\wedge b\ge 0\right)\Rightarrow {\Sigma }^{-1}b\ge 0$. Hence, just as clearly, ${\Sigma }^{-1}b\notin -C$. Since ${\mathbb{R}}_{+}^n$ is a self-dual closed convex cone, one has $C^{*}={\mathbb{R}}_{+}^n$ and, thus, ${\Sigma }^{-1}b\in C^{*}$. Since ${\Sigma }^{-1}b\in C^{*}\Rightarrow b\in \Sigma \left(C^{*}\right)$, we conclude that $b\in \Sigma \left(C^{*}\right)\setminus -C$ as required.    □

4. Some Applications

In this section, we discuss some applications of Theorem 2 and its corollary as well as of Theorem 1 in the perspective of the duality of cones and polyhedra. Indeed, we have already noticed that the minimization problem associated with the bounds given in the previous section reduces to minimizing the positive-definite quadratic form $q\left(u\right)=\langle u,\Sigma \left(u\right)\rangle$ over $\mathfrak{a}(b,C)$ or over $\mathfrak{b}\left(T\right)$. Even if $T=b+C$ and C is a closed convex cone, this minimization problem is a non-trivial convex programming problem [15]. However, knowing the structure of $\mathfrak{b}\left(T\right)$ allows us to easily compute upper bounds on the probability of certain interesting tail events.

4.1. Sampling Positive-Definite Matrices from a Random Ensemble

A random vector X in the Euclidean space $\mathcal{E}=(V,\langle ·,·\rangle )$ with a covariance function $\Sigma$ has a weakly spherical distribution if the distribution of X is such that $\Sigma ={\sigma }^2{\mathrm{id}}_V$, where ${\mathrm{id}}_V$ is the identity endomorphism of V and ${\sigma }^2$ is a positive real number [16]. A square random matrix is a just a random vector in $({\mathbb{R}}^{n\times n},{\langle ·,·\rangle }_F)$, where ${\langle X,Y\rangle }_F=\mathrm{Tr}\left(X^{\prime }Y\right)$ is the Frobenius inner product. A symmetric random matrix is, thus, a random vector in $(S^n,{\langle ·,·\rangle }_F)$, where $S^n$ is the real vector space of real symmetric matrices. Several models (i.e., probability measures) of symmetric random matrices are known and well understood [17]. Probably the best-known example of such models is the Symmetric Wigner Ensemble, which includes the Gaussian Orthogonal Ensemble. A random matrix is a symmetric Wigner matrix (or sampled from a symmetric Wigner ensemble) if its diagonal entries are independently and identically distributed (i.i.d.) according to a given distribution and the off-diagonal elements in the upper triangular part are i.i.d. according to a given distribution that may differ from the former. A symmetric real ensemble is, thus, completely specified by the distribution of the entries of the matrix and, thus, does not depend on the order of the matrix. If the diagonal elements of a symmetric Wigner matrix W follow a standard normal distribution, while the off-diagonal elements follow a normal distribution with zero mean and variance $1/2$, then W is a matrix from the Gaussian Orthogonal Ensemble. Here is one of the simplest mechanisms that, given a random matrix ensemble, yields a symmetric random matrix ensemble: if M is a random matrix whose entries are i.i.d. random variables distributed according to the law P, then $S={\displaystyle \frac{M+M^{\prime }}{2}}$ is a symmetric Wigner matrix—taking $M^{\prime }M$ for S instead yields the symmetric Wishart ensemble. If the first two moments of P are 0 and ${\sigma }^2$, respectively, for some positive real number $\sigma$, then we denote the resulting ensemble by $\mathcal{W}\left(\sigma \right)$. By construction, any matrix S sampled from $\mathcal{W}\left(\sigma \right)$ has a weakly spherical distribution. To see this, note that the variance of each diagonal entry of S is ${\sigma }^2$, while the variance of each off-diagonal entry of S is ${\sigma }^2/2$, since $S=(M+M^{\prime })/2$ by construction. Next, if $\left(E_{i,j}|i\le j\right)$ is the standard basis of $S^n$, then, by the definition of the covariance function of random vectors in Euclidean spaces (see the previous section), one has

$$\mathbb{E}\left({\langle E_{i,j},S\rangle }_F{\langle E_{h,k},S\rangle }_F\right)=\mathbb{E}\left(\mathrm{Tr}\left(E_{i,j}S\right)\mathrm{Tr}\left(E_{h,k}S\right)\right)=\left\{\begin{array}{cc}\mathbb{E}\left(S_{i,i}^2\right)\hfill & \mathrm{if}i=j=h=k\hfill \\ 4\mathbb{E}\left(S_{i,j}^2\right)\hfill & \mathrm{if}\{i,j\}=\{h,k\},i\ne j\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

Therefore, since $\mathbb{E}\left(S_{i,i}^2\right)={\sigma }^2$ and $\mathbb{E}\left(S_{i,j}^2\right)={\sigma }^2/2$, it follows that

$$\mathbb{E}\left({\langle E_{i,j},S\rangle }_F{\langle E_{h,k},S\rangle }_F\right)={\sigma }^2{\langle E_{i,j},E_{h,k}\rangle }_F={\langle E_{i,j},{\sigma }^2{\mathrm{id}}_{S^n}\left(E_{h,k}\right)\rangle }_F.$$

One of the most fundamental problems in random matrix theory is to understand the spectrum of a random matrix from a given ensemble, at least asymptotically, namely, when the order of the matrix goes to ∞. Even more fundamental is to compute the probability that a random matrix X is positive-definite (or negative-definite), i.e., that its smallest eigenvalue ${\lambda }_n\left(X\right)$ is positive. This problem is completely and precisely solved for Gaussian matrices (see [4,5,6]): the probability of the event ${\lambda }_n\left(X\right)>0$ when X is sampled from a symmetric Gaussian ensemble goes to zero as $e^{-c{n}^2}$, where $c=ln3/4$. Using corollary 1, we prove here that the same is true in general, although the rate we can provide is much weaker than $e^{-c{n}^2}$.

Lemma 8. 

Let σ be a positive real number and ${\lambda }_n\left(X\right)$ be the least eigenvalue of the random matrix X sampled from $\mathcal{W}\left(\sigma \right)$. For any positive real number α and any positive number ξ, if $n\ge {\sigma }^2/{\xi }^{2+\alpha }$, then $\mathrm{Pr}\left\{{\lambda }_n\left(X\right)\ge \xi \right\}\le {\xi }^{\alpha }$.

Proof. 

Since the statement trivially holds when $\xi \ge 1$, we may assume that $\xi <1$. The event ${\lambda }_n\left(X\right)\ge \xi$ is the same as the event $X{⪰}_{S_{+}^n}\xi I_n$, where $I_n$ is the identity matrix of order n and $S_{+}^n$ denotes the cone of positive semi-definite matrices lying in $\mathcal{E}=(S^n,{\langle ·,·\rangle }_F)$. As observed above, X has a weakly spherical distribution with a covariance function $\Sigma$ of the form ${\sigma }^2{\mathrm{id}}_{S^n}$. It is well known that $S_n^{+}$ is a self-dual closed convex cone [10]. Hence, ${\left(S_n^{+}\right)}^{*}=S_n^{+}$ and

$$\Sigma \left({\left(S_n^{+}\right)}^{*}\right)=\Sigma \left(S_n^{+}\right)={\sigma }^2{\mathrm{id}}_{S^n}\left(S_n^{+}\right)=S_n^{+}.$$

Since $\xi I_n\in S_{+}^n$, it follows that the hypotheses of Corollary 1 are satisfied by X with $b=\xi I_n$ and $C=S_{+}^n$. Therefore, after observing that $\parallel \xi I_n{\parallel }_{{\Sigma }^{-1}}^2={\displaystyle \frac{n{\xi }^2}{{\sigma }^2}}$, it follows by (8) that

$$\mathrm{Pr}\left\{{\lambda }_n\left(X\right)\ge \xi \right\}=\mathrm{Pr}\left\{X{⪰}_{S_{+}^n}\xi I_n\right\}\le {\displaystyle \frac{1}{1+\parallel \xi I_n{\parallel }_{{\Sigma }^{-1}}^2}}={\displaystyle \frac{{\sigma }^2}{{\sigma }^2+n{\xi }^2}}\le {\xi }^{\alpha },$$

where the last inequality is implied by $n\ge {\sigma }^2/{\xi }^{2+\alpha }$.    □

Theorem 3. 

Let σ be a positive real number and ${\lambda }_n\left(X\right)$ be the least eigenvalue of the random matrix X sampled from $\mathcal{W}\left(\sigma \right)$. It holds that ${lim}_{n\to \infty }\mathrm{Pr}\left\{{\lambda }_n\left(X\right)>0\right\}=0$.

Proof. 

For any non-negative real number y, let ${\ell }_n\left(y\right)=\mathrm{Pr}\left\{{\lambda }_n\left(X\right)\ge y\right\}$. If $\mathrm{Pr}\left\{{\lambda }_n\left(X\right)>0\right\}$ does not converge to zero, then there is a real number $\xi$ and a sub-sequence ${\left\{{\ell }_{n_{\nu }}\left(\xi \right)\right\}}_{\nu \in \mathbb{N}}$ such that $0<\xi <1$ and ${lim}_{\nu \to \infty }{\ell }_{n_{\nu }}\left(\xi \right)=\beta$ for some $\beta$ such that $0<\beta <1$. Hence, for any $ϵ\in \mathbb{R}$ such that $0<ϵ<\beta$, there is a ${\nu }_0\in \mathbb{N}$ such that

$${\ell }_{n_{\nu }}\left(\xi \right)\ge \beta -ϵ\forall \nu \ge {\nu }_0.$$

On the other hand, by Lemma 8, for every positive real number $\alpha$, it holds that

$${\ell }_n\left(\xi \right)\le {\xi }^{\alpha }\forall n\ge {\displaystyle \frac{{\sigma }^2}{{\xi }^{2+\alpha }}}.$$

Let

$${\alpha }_0={\displaystyle \frac{ln(\beta -ϵ)}{ln\left(\xi \right)}},$$

where ln is the natural logarithm. Since $\xi$ and $\beta -ϵ$ are both positive and strictly smaller than 1, it holds that ${\alpha }_0>0$. Hence, if $\alpha >{\alpha }_0$, then ${\xi }^{\alpha }<\beta -ϵ$. If ${\nu }_1$ is such that $n_{{\nu }_1}>{\displaystyle \frac{{\sigma }^2}{{\xi }^{2+\alpha }}}$, then for $\nu \ge max\{{\nu }_0,{\nu }_1\}$, it holds that

$$\beta -ϵ<{\ell }_{n_{\nu }}\left(\xi \right)\le {\xi }^{\alpha }$$

But this contradicts ${\xi }^{\alpha }<\beta -ϵ$, which holds by the choice of $\alpha$. This contradiction proves the theorem.    □

4.2. Tails of Linear Images of Random Vectors and Asymptotic Feasibility of Systems of Linear Inequalities

Concentration and tail inequalities for linear, possibly random images of random vectors are a challenging topic in probability and statistics [7,8]. Let X be a random vector in ${\mathcal{R}}^n$ and f be the linear map $X\mapsto AX$, where $A\in {\mathbb{R}}^{m\times n}$ is a given matrix. According to [7], a typical tail inequality is an inequality of the form

$$\mathrm{Pr}\left\{f\left(X\right){⪰}_{{\mathbb{R}}^m}sb\right\}\le g\left(s\right)$$

for some function $g:{\mathbb{R}}_{+}\to [0,1]$. See [8] for the related problem of determining general concentration inequalities. Notably, Ref. [7] investigates the more general case of the map $\mathcal{A}X$, where $\mathcal{A}$ is a random matrix and X is a random vector, under certain hypotheses on the distribution of $\mathcal{A}$, assuming that $\mathcal{A}$ and X are independent. If the matrix f is a deterministic linear map and X has a positive-definite covariance function $\Sigma$, then, as we will show in the next theorem, there always exists a rather general and easy-to-compute upper bound $g\left(s\right)$ on the probability measure of the tails of $f\left(X\right)$. In Theorem 5, on the other hand, we will exhibit a transparent relation between the ordinary tails of the linear images of a random vector X and the generalized tails of X.

Theorem 4. 

Let $A\in {\mathbb{R}}^{m\times n}$ be a real matrix, $b={(b_1,\dots ,b_m)}^{\prime }$ be a real vector, and $b^{+}\in {\mathbb{R}}^m$ be defined component-wise by $b_i^{+}=max(0,b_i)$, $i=1\dots ,m$. If X is a centered random vector with a covariance matrix ${\sigma }^2{I}_n$ and $b^{+}\ne 0$, then

$$\mathrm{Pr}\left\{AX\ge sb\right\}\le {\displaystyle \frac{{\parallel A\parallel }_{\mathrm{op}}^2}{{\parallel \frac{s}{\sigma }{b}^{+}\parallel }^2+{\parallel A\parallel }_{\mathrm{op}}^2}},$$

(9)

where $s\ge 0$ is a real number and $\parallel ·\parallel$ and ${\parallel ·\parallel }_{\mathrm{op}}$ are the Euclidean and the operator norm—induced by Euclidean norms—respectively.

Proof. 

Let $T\left(s\right)=\{x\in {\mathbb{R}}^n|Ax\ge sb\}$, and note that $T\left(s\right)$ is a possibly empty polyhedron. If $s=0$, the result is trivial because the right side of (9) is 1 in this case. If $T\left(s\right)$ is empty, the result is also trivial because the left-hand side of (9) is zero. So, we assume that $s>0$ and $T\left(s\right)$ is not empty. Obviously, $T\left(s\right)=\{x\in {\mathbb{R}}^n|A\left(s\right)x\ge b\}$, where $A\left(s\right)=s^{-1}A$ for $s>0$. Since $b^{+}\ne 0$, it follows that $T\left(s\right)$ does not contain the origin of ${\mathcal{R}}^n$. Hence,

$$\mathrm{Pr}\left\{AX\ge sb\right\}=\mathrm{Pr}\left\{X\in T\left(s\right)\right\}\le \underset{u\in \mathfrak{b}\left(T\right(s\left)\right)}{inf}{\displaystyle \frac{{\parallel \sigma u\parallel }^2}{1+{\parallel \sigma u\parallel }^2}}$$

by Theorem 1 with $\Sigma ={\sigma }^2{I}_n$. Let us compute the blocker $\mathfrak{b}\left(T\right(s\left)\right)$ of $T\left(s\right)$. Let $c:{\mathbb{R}}^n\to \mathbb{R}$ be defined by

$$u\mapsto c\left(u\right)=min\left\{u^{\prime }x\right|x\in T\left(s\right)\};$$

that is, $c\left(u\right)$ is the minimum of a linear program over the non-empty polyhedron $T\left(s\right)$ and the minimum might be $-\infty$. By linear programming duality, it holds that $c\left(u\right)=max\left\{v^{\prime }b\right|A^{\prime }\left(s\right)v=u,v\in {\mathbb{R}}_{+}^m\}$. Since, by definition,

$$u\in \mathfrak{b}\left(T\left(s\right)\right)⟺u^{\prime }x\ge 1\forall x\in T\left(s\right)⟺c\left(u\right)\ge 1,$$

it follows that

$$\mathfrak{b}\left(T\left(s\right)\right)=\left\{A^{\prime }\left(s\right)v\right|v^{\prime }b\ge 1,v\in {\mathbb{R}}_{+}^m\}.$$

Therefore, if $\overline{v}$ is any vector in ${\mathbb{R}}_{+}^m$ such that ${\overline{v}}^{\prime }b\ge 1$, then

$$\underset{u\in \mathfrak{b}\left(T\right(s\left)\right)}{inf}{\displaystyle \frac{{\parallel \sigma u\parallel }^2}{1+{\parallel \sigma u\parallel }^2}}=\underset{\begin{array}{c}v\in {\mathbb{R}}_{+}^m\\ v^{\prime }b\ge 1\end{array}}{inf}{\displaystyle \frac{{\parallel \sigma A\left(s\right)}^{\prime }{v\parallel }^2}{{1+\parallel \sigma A\left(s\right)}^{\prime }{v\parallel }^2}}\le {\displaystyle \frac{{\parallel \sigma A\left(s\right)}^{\prime }\overline{v}{\parallel }^2}{{1+\parallel \sigma A\left(s\right)}^{\prime }\overline{v}{\parallel }^2}}.$$

Since the map $\xi \mapsto {\displaystyle \frac{\xi }{1+\xi }}$ defines a strictly increasing function over ${\mathbb{R}}^{+}$ and since

$${\parallel \sigma A\left(s\right)}^{\prime }\overline{v}{\parallel }^2={\left({\displaystyle \frac{\sigma }{s}}\right)}^2\parallel A^{\prime }\overline{v}{\parallel }^2\le {\left({\displaystyle \frac{\sigma }{s}}\right)}^2\parallel A^{\prime }{\parallel }_{\mathrm{op}}^2{\parallel \overline{v}\parallel }^2,$$

it follows that

$$\underset{u\in \mathfrak{b}\left(T\right(s\left)\right)}{inf}{\displaystyle \frac{{\parallel \sigma u\parallel }^2}{1+{\parallel \sigma u\parallel }^2}}\le {\displaystyle \frac{{\parallel A\parallel }_{\mathrm{op}}^2{\parallel \overline{v}\parallel }^2}{{\left(\frac{s}{\sigma }\right)}^2+{\parallel A\parallel }_{\mathrm{op}}^2{\parallel \overline{v}\parallel }^2}}.$$

To complete the proof, choose ${\displaystyle \frac{b^{+}}{\parallel b^{+}{\parallel }^2}}$ for $\overline{v}$ and check that ${\displaystyle \frac{b^{+}}{\parallel b^{+}{\parallel }^2}}\ge 0$ and that $b^{\prime }{\displaystyle \frac{b^{+}}{\parallel b^{+}{\parallel }^2}}=1$.    □

Some remarks are in order.

Remark 2. 

If X has a positive covariance matrix Σ, then the theorem is easily extended: just replace ${\parallel A\parallel }_{\mathrm{op}}$ with $\parallel A{\Sigma }^{1/2}{\parallel }_{\mathrm{op}}$ and set $\sigma =1$.

Remark 3. 

If $T\left(s\right)$ is not full-dimensional and X has an absolutely continuous distribution with respect to the Lebesgue measure, then the result is useless because $T\left(s\right)$ is a null set; however, the result becomes interesting if either $T\left(s\right)$ has positive Lebesgue measure or X has a singular distribution with respect to the Lebesgue measure.

Remark 4. 

If $b\ge 0$, so $b=b^{+}$, then $T\left(s\right)\supseteq T\left(s^{\prime }\right)$ for $s\le s^{\prime }$. Thus, if ${\left(a_n\right)}_{n\in \mathbb{N}}$ is any strictly positive strictly increasing sequence, then ${\left(T\left(a_n\right)\right)}_{n\in \mathbb{N}}$ is an (inclusion-wise) decreasing sequence of polyhedra; the theorem says no more than we expect in this case. However, this is no longer true if $b\ne b^{+}$. Indeed, let A and b be as follows

$$A=\left(\begin{array}{cc}\hfill -1& 1\\ \hfill -1& -1\\ \hfill 1& 0\end{array}\right),b=\left(\begin{array}{c}\hfill 1\\ \hfill -2\\ \hfill -1\end{array}\right).$$

With this choice of A and b, $T\left(s\right)$ is the triangle in ${\mathcal{R}}^2$ whose vertices are

$$\alpha \left(s\right)=\left(\begin{array}{c}\hfill -s\hfill \\ \hfill 0\hfill \end{array}\right),\beta \left(s\right)=\left(\begin{array}{c}\hfill -s\hfill \\ \hfill 3s\hfill \end{array}\right),\gamma \left(s\right)=\left(\begin{array}{c}\hfill s/2\hfill \\ \hfill 3s/2\hfill \end{array}\right),$$

and we see that $\{\alpha \left(s\right),\gamma \left(s\right)\}\subseteq T\left(s\right)\setminus T\left(s^{\prime }\right)$ for all $s^{\prime }>s$, while the area of $T\left(s\right)$ is ${\left({\displaystyle \frac{3s}{2}}\right)}^2$, which is clearly increasing in s.

Remark 5. 

Theorem 4 can be used to study the asymptotic probability of feasibility of the system $T=\{x\in {\mathbb{R}}^n|Ax\ge b\}$ when A and b grow in some sense. More precisely, identify T with the pair $(A,b)$ and consider the sequence ${\left\{(A_{\nu },b_{\nu })\right\}}_{\nu \in \mathbb{N}}$ of linear systems $T_{\nu }$ with $A_{\nu }\in {\mathbb{R}}^{m\left(\nu \right)\times n\left(\nu \right)}$ and $b_{\nu }\in {\mathbb{R}}^{m\left(\nu \right)}$, where the number of inequalities and the number of unknowns are both integer (possibly constant) functions of the index ν; also, let $X_{\nu }$ be a sequence of random vectors such that $X_{\nu }$ is in ${\mathcal{R}}^{n\left(\nu \right)}$ and $X_{\nu }$ has a covariance matrix ${\Sigma }_{\nu }={\sigma }^2{I}_{n\left(\nu \right)}$. Finally, let $p_{\nu }=\mathrm{Pr}\left\{A_{\nu }{X}_{\nu }\ge b_{\nu }\right\}$ be the probability that the system $T_{\nu }$ is feasible under the probability measure induced by $X_{\nu }$. A straightforward consequence of Theorem 4 is that if ${\parallel A_{\nu }\parallel }_{\mathrm{op}}=o(\parallel b_{\nu }^{+}\parallel )$, then $p_{\nu }=o\left(1\right)$. The condition $\parallel A_{\nu }{\parallel }_{\mathrm{op}}=o(\parallel b_{\nu }^{+}\parallel )$ can be realized in several ways, either by keeping $m\left(\nu \right)$ and $n\left(\nu \right)$ constant or by letting them grow. Let us give a theoretical graph example in the latter case. Let G be a simple undirected graph with n vertices $\{v_1,\dots ,v_n\}$. For two vertices $v_i,v_j\in V\left(G\right)$, $i\ne j$, write $v_i\sim v_j$ whenever $v_i$ ad $v_j$ are adjacent in G. If $v_i\sim v_j$, then we say that $v_i$ ad $v_j$ are neighbors. Let $d_i$ denote the degree of vertex $v_i$, namely, the number of neighbors of $v_i$ in G. Also, denote, by $\Delta \left(G\right)$, the maximum of the degrees of G. The Laplacian matrix $L\left(G\right)$ of G is the order n matrix defined by

$${\left(L\left(G\right)\right)}_{i,j}=\{\begin{array}{cc}\hfill \phantom{-}{d}_i& if i=j\hfill \\ \hfill -1& if i\ne j and v_i\sim v_j.\hfill \end{array}$$

Recall that for functions $h,k:\mathbb{N}\to \mathbb{N}$, $k\left(n\right)=\Omega \left(h\right(n\left)\right)$ means that $h\left(n\right)=O\left(k\right(n\left)\right)$. Let ${\left(G_{\nu }\right)}_{\nu }$ be a sequence of graphs on $n\left(\nu \right)$ vertices, and let ${\left(L_{\nu }\right)}_{\nu }$ be the corresponding sequences of Laplacians. Also, let ${\Delta }_{\nu }:=\Delta \left(G_{\nu }\right)$. It is a classical result of Fiedler [18] that the spectral radius of $L_{\nu }$ is, at most, $2{\Delta }_{\nu }$, so ${\parallel A_{\nu }\parallel }_{\mathrm{op}}\le 2{\Delta }_{\nu }=O\left({\Delta }_{\nu }\right)$. Hence, if ${\Delta }_{\nu }=o\left(\sqrt{n}\right)$ and $b_{\nu }$ is a dense $\{-1,0,1\}$ vector, say with $\Omega \left(n\right(\nu \left)\right)$ non-zero entries, such that $\parallel b_{\nu }^{+}\parallel =\Omega (\parallel b_{\nu }\parallel )$, then the condition ${\parallel A_{\nu }\parallel }_{\mathrm{op}}=o(\parallel b_{\nu }^{+}\parallel )$ is satisfied when $n\left(\nu \right)$ grows with ν.

The following result relates more transparently the ordinary tails of the linear image of a random vector X in ${\mathcal{R}}^n$ to the generalized tails of the same vector. Recall that a matrix $A\in {\mathbb{R}}^{m\times n}$ is right-invertible if there exists a matrix $A^{‡}\in {\mathbb{R}}^{n\times m}$ such that $A{A}^{‡}=I_m$ with I being the identity matrix of order m. If A has full row rank, then the Moore–Penroose pseudo-inverse $A^{†}$ can be taken for $A^{‡}$, where $A^{†}=A^{\prime }{\left(A{A}^{\prime }\right)}^{-1}$. A linear map $f:{\mathbb{R}}^n\to {\mathbb{R}}^m$ is right-invertible if the matrix representing f with respect to the canonical basis of ${\mathbb{R}}^n$ and ${\mathbb{R}}^m$ is such. Note that a right-invertible map is surjective.

Theorem 5. 

Let $A\in {\mathbb{R}}^{m\times n}$ be a real matrix and $b\in {\mathbb{R}}^m$ be a real vector. Let $f:{\mathbb{R}}^n\to {\mathbb{R}}^m$ be the linear map $u\mapsto Au$ and let $C_A=\{u\in {\mathbb{R}}^n|Au\ge 0\}$. If X is a random vector in ${\mathcal{R}}^n$ and f is right-invertible, then

$$\mathrm{Pr}\left\{f\left(X\right)\ge b\right\}=\mathrm{Pr}\left\{X{⪰}_{C_A}{f}^{‡}\left(b\right)\right\},$$

where $f^{‡}$ is the linear map induced by $A^{‡}$.

Proof. 

First of all, note that $C_A$ is a polyhedral cone. So, $C_A$ is a closed convex cone. However, unless f is injective, $C_A$ is not a proper cone. Indeed, if f is not injective, then its kernel is a non-trivial subspace of ${\mathcal{R}}^n$ contained in $C_A$. However, $C_A$ induces a pre-order, and so, $X{⪰}_{C_A}{f}^{‡}\left(b\right)$ is a well-defined generalized tail. It remains to be proven that the stated equality holds. This fact follows easily since $f\left(X-f^{‡}\left(b\right)\right)=f\left(X\right)-b$. So, $f\left(X\right)-b\in {\mathbb{R}}_{+}^m\iff \left(X-f^{‡}\left(b\right)\right)\in C_A$.    □

4.3. Testing Homophily in Randomly Colored Networks

Tail inequalities, albeit in a different guise, have been successfully used in [9] to evaluate what is called network homophily. Suppose we have a simple undirected graph on the vertex set V, a surjective map $f:V\to \{1,\dots ,s\}$, which can be thought of as coloring of the vertices, and a coloring profile $\mathbf{c}=(c_1\cdots ,c_s)$, where $c_i$ is the number of vertices in $f^{-1}\left(i\right)$, $i=1\dots ,s$, i.e., $c_i$ is the number of vertices of color i. With these data, we want to evaluate whether the graph is homophilic, i.e., whether it is true that the s-tuple $m=(m_1,\dots ,m_s)$, where $m_i$ is the number of edges of the subgraph induced by the vertices of color i, is, in some sense, larger than what we would expect if the colors did not correlate with the adjacency of vertices—recall that an edge e of a graph G is induced by a subset U of the set of vertices $V\left(G\right)$ if both the end vertices of e are in U. The approach taken in [9] is to define the simple probability space $({\Phi }_{\mathbf{c}},{\mathbb{P}}_{\mathbf{c}})$—called the random coloring model—where ${\Phi }_{\mathbf{c}}$ is the set of all surjective maps $g:V\to \{1,\dots ,s\}$ such that $g^{-1}\left(i\right)=c_i$, $i=1\dots ,s$, and ${\mathbb{P}}_{\mathbf{c}}$ is the uniform measure $c_1!\dots c_s!/n!$ on ${\Phi }_{\mathbf{c}}$. In the random coloring model, m is just the observed value of the random vector $M={(M_1\dots ,M_s)}^{\prime }$, where $M_i$ is the number of edges induced by the vertices of color i in a random coloring of V of G. Moreover, both the mean vector ${\mu }_{\mathbf{c}}$ and the covariance matrix ${\Sigma }_{\mathbf{c}}$ of M can be given in closed form under the random coloring model. Thus, if $\varphi :{\mathbb{R}}^s\to {\mathbb{R}}_{+}$ is a linear function—e.g., the arithmetic mean—one can evaluate the homophily of the input graph by considering the one-sided tail probability $\mathrm{Pr}\left\{\varphi (M-\mu )\ge \varphi (m-\mu )\right\}$: any good upper bound on this tail probability can quantify the homophily of the given network by measuring, at least approximately, in terms of $\varphi$ how significant the deviation of m from $\mu$ is. In particular, if $\varphi (m-\mu )>0$—otherwise, the network is declared not homophilic at all—one can apply the classical one-dimensional Cantelli inequality (1) to estimate the approximate significance of the deviation of the observed value from the expected one. Note that the linear map $\varphi$, say the arithmetic mean, acts as a global synthesis of network homophily but cannot identify hypodense or poorly significant color classes. We show here, as a consequence of Corollary 2, that generalized tails do not suffer from these limitations. To see this, for a set $R\subseteq \{1,\dots ,s\}$, let $\overline{R}=\{1,\dots ,s\}-R$ and $W_{\overline{R}}={⨁}_{\overline{R}}\mathbb{R}{e}_i$, where $e_i$ is the i-th fundamental vector of ${\mathbb{R}}^s$, $i=1\dots ,s$. Thus, $W_{\overline{R}}$ is the subspace spanned by the coordinates whose index is in $\overline{R}$. The projection ${\pi }_R:{\mathbb{R}}^s\to W_{\overline{R}}$ of ${\mathbb{R}}^s$ onto ${\mathbb{R}}^s/W_{\overline{R}}$ takes the vector u to the vector $u_R$, suppressing the coordinates whose index is not in R—if $s=4$ and $R=\{1,3\}$, then $u_R={(u_1,u_3)}^{\prime }$. Also, recall that a graph G is said to be over-dispersed if the ratio of the variance to the mean of its degree distribution is greater than 1. Most protein–protein interaction networks are over-dispersed.

Theorem 6. 

Let $(G,f)$ be an over-dispersed colored graph where the coloring $f:V\to \{1,\dots ,s\}$ has the profile $\mathbf{c}=(c_1,\dots ,c_s)$, $c_i\ge 2$, $i=1,\dots ,s$. Let m, M, ${\mu }_{\mathbf{c}}$ and ${\Sigma }_{\mathbf{c}}$ be defined as above, and let $x=m-\mu$, $X=M-\mu$, and

$$S\left(m\right)=\left\{i\in \{1,\dots ,s\}\left\}\right|{\mu }_i<m_i<\left({\displaystyle \genfrac{}{}{0pt}{}{c_i}{2}}\right)\right\}.$$

Also, for any $R\subseteq S\left(m\right)$, let ${\mu }_R$ and ${\Sigma }_R$ be the mean vector and the covariance matrix of the random vector $X_R$ image of the projection of X onto $W_{\overline{R}}$. If $S\left(m\right)\ne \varnothing$, then for any $R\subseteq S\left(m\right)$, it holds that

$$\mathrm{Pr}\left\{M_R\ge m_R\right\}\le {\displaystyle \frac{1}{1+\parallel m_R-{\mu }_R{\parallel }_{{\Sigma }_R^{-1}}^2}},$$

(10)

where $M_R$ and $m_r$ are the projection of M and m onto $W_{\overline{R}}$, respectively.

Proof. 

Let ${\sigma }_i^2$ denote the variance of $M_i$, the i-th coordinate of M, where $i=1\dots ,s$. Fix $R\subseteq S\left(m\right)$ and observe that $M_R\ge m_R\iff X_R-x_R\in {\mathbb{R}}^r$, where r is the cardinality of R. Furthermore, it can be shown that ${\mu }_R=\mathbb{E}\left({\pi }_R\left(X\right)\right)={\pi }_R\left(\mathbb{E}\left(X\right)\right)$ and that ${\Sigma }_R$ is the submatrix of ${\Sigma }_{\mathbf{c}}$ obtained by suppressing the rows and the columns whose index is not in R. Since $x_R⪈0$ and $X_R$ is a centered random vector, it follows that the hypotheses of Corollary 2 are met if ${\Sigma }_R$ is a monotone matrix. If ${\Sigma }_R$ is monotone, then, by the corollary, $\mathrm{Pr}\left\{X_R\ge x_R\right\}\le (1+\parallel x_R{{\parallel }_M^2)}^{-1}$, where $\parallel x_R{\parallel }_M={\parallel x_R\parallel }_{{\Sigma }_R^{-1}}$ is the so-called Mahalanobis norm of $x_R$. It remains to be shown that if G is an over-dispersed graph, then, for every $R\subseteq S\left(m\right)$, ${\Sigma }_R$ is a monotone matrix, which means showing that ${\Sigma }_R$ is invertible (hence, positive-definite) and ${\Sigma }_R^{-1}\ge 0$. According to the results in [9], the covariance matrix ${\Sigma }_{\mathbf{c}}$ is of the form ${\Sigma }_{\mathbf{c}}=D+\lambda \left(G\right)u{u}^{\prime }$, where $u={(u_1,\dots ,u_s)}^{\prime }$ is such that $u_i>0$ for $i=1,\dots ,s$, D is a diagonal matrix whose i-th diagonal entry is ${\sigma }_i^2-\lambda \left(G\right)u_i^2$, and $\lambda \left(G\right)\le 0$ (because G is over-dispersed). Clearly, ${\Sigma }_R=D_R+\lambda \left(G\right)u_R{u}_R^{\prime }$, where $D_R$ is the sub-matrix of D obtained by suppressing the rows and the columns whose index is not in R. Since the off-diagonal elements of ${\Sigma }_R$ are non-positive, ${\Sigma }_R$ is a Z-matrix, and it is known (see [14] Chapter 6) that if a Z-matrix is positive-definite, then it is monotone. On the other hand, since ${\Sigma }_R$ is a covariance matrix, it follows that ${\Sigma }_R$ is positive semidefinite. Thus, to show that ${\Sigma }_R$ is monotone, it is sufficient to show that ${\Sigma }_R$ is non-singular. By the Sherman–Morrison formula, if $D_R$ is non-singular, then ${\Sigma }_R$ is also non-singular. Therefore, to prove that ${\Sigma }_R$ is monotone, we simply check that ${\sigma }_i^2-\lambda \left(G\right)u_i^2\ne 0$ for $i\in S\left(m\right)$. Note that if $\lambda \left(G\right)\ne 0$, then ${\sigma }_i^2-\lambda \left(G\right)u_i^2>0$ because $u_i^2>0$ and $\lambda \left(G\right)<0$. So, the only possibility is that $\lambda \left(G\right)=0$ and ${\sigma }_i^2=0$. However, by Theorem 1 in [19], ${\sigma }_i^2=0$ if and only if $m_i\in \{0,\left({\displaystyle \genfrac{}{}{0pt}{}{c_i}{2}}\right)\}$, which cannot happen because of the definition of $S\left(m\right)$, using the fact that ${\mu }_i\ge 0$, being that ${\mu }_i$ is the expected value of the non-negative random variable $M_i$. So, we conclude that ${\Sigma }_R^{-1}\ge 0$ for all $R\subseteq S\left(m\right)$, and this completes the proof.    □

We conclude that under the random coloring model, the Mahalanobis norm of $m-\mu$ is a directed measure of the statistical significance of the deviation of m from $\mu$, and thus, it can be taken as a measure of the joint homophily of the vertices whose color is in $S\left(m\right)$.

After an observation in [9], the random coloring model allows us to give a precise formalization of how difficult it can be to provide bounds on tail probabilities. First, recall that a subset I of vertices of a graph H is an independent set of H if I induces no edges. Also, recall that the Independent Set Problem, INDEPENDENT SET, is the problem of deciding whether, given a graph H and a non-negative integer k, $V\left(H\right)$ contains an independent set of at least k vertices. If H has an independent set of at least k vertices, we say that $(H,k)$ is a YES instance of INDEPENDENT SET. INDEPENDENT SET is an archetypal hard problem, which means that it is in the NP-complete class (see [20], Section 3.1.3). It is widely believed that it cannot be decided in polynomial time whether instances of problems in such a class are YES instances. Next, let $X=M_i$ be the i-th coordinate of M and $\kappa =c_i$ be the i-th coordinate of the coloring profile $\mathbf{c}$. Without the loss of generality, $i=1$ (the argument we will use is independent of i by symmetry). With this notation, $\mathrm{Pr}\left\{X\ge 1\right\}$ is the marginal tail probability $\mathrm{Pr}\left\{M_1\ge 1\right\}$. By definition, X is the number of edges induced by a set of $\kappa$ vertices sampled without repetition from the vertex set of the input graph G.

Theorem 7. 

Let X and κ be as above. If G is the input graph in the random coloring model, then $\mathrm{Pr}\left\{X\ge 1\right\}<1$ if and only if $(G,\kappa )$ is a YES instance of INDEPENDENT SET.

Proof. 

If $(G,\kappa )$ is a YES instance of INDEPENDENT SET, then G has an independent set of at least $\kappa$ vertices. Hence, $\mathrm{Pr}\left\{X=0\right\}>0$ because such a set is sampled with positive probability. Since $\mathrm{Pr}\left\{X=0\right\}>0\iff \mathrm{Pr}\left\{X\ge 1\right\}<1$, the only if part follows. Conversely, if $\mathrm{Pr}\left\{X\ge 1\right\}<1$, then $\mathrm{Pr}\left\{X=0\right\}>0$. Therefore, the event $X=0$ occurs with positive probability (i.e., 0 is in support of X). Since $X=0$ occurs if and only if $(G,\kappa )$ is a YES instance of INDEPENDENT SET, the if part follows as well.    □

In other words, in general, even deciding the sign of $1-\mathrm{Pr}\left\{X\ge 1\right\}$ for a finitely supported random variable X is a computationally hard problem.

5. Concluding Remarks

Tail inequalities are a challenging problem in probability and statistics. According to Theorem 7, we can expect no more than easily computable bounds for dealing with tail probabilities, even if we have no control over the quality of the approximation. One such bound is given by Cantelli’s inequality (1). This bound was significantly extended by Marshall and Olkin (see Theorem 1) from random variables taking values on the real line to random vectors taking values in standard real n-dimensional Euclidean spaces. By looking at the inequality from the perspective of convex cone duality, as shown in Section 3, this paper naturally extends the inequality to any finite-dimensional Euclidean space. The new perspective allows us to cast (and bound) the probability of certain interesting events arising in random matrix theory, the theory of tail and concentration inequalities for linear images of random vectors, and network homophily as the probability of generalized tails, namely, inequalities induced by the pre-order (or the partial order) defined by convex cones. Besides Cantelli inequalities, there are several other tail inequalities. It would certainly be worthwhile to check whether some of these one-dimensional inequalities have a corresponding generalized version.