Information Properties of a Random Variable Decomposition through Lattices ^†

Abstract

A full-rank lattice in the Euclidean space is a discrete set formed by all integer linear combinations of a basis. Given a probability distribution on ${\mathbb{R}}^n$, two operations can be induced by considering the quotient of the space by such a lattice: wrapping and quantization. For a lattice $\Lambda$, and a fundamental domain $\mathcal{D}$, which tiles ${\mathbb{R}}^n$ through $\Lambda$, the wrapped distribution over the quotient is obtained by summing the density over each coset, while the quantized distribution over the lattice is defined by integrating over each fundamental domain translation. These operations define wrapped and quantized random variables over $\mathcal{D}$ and $\Lambda$, respectively, which sum up to the original random variable. We investigate information-theoretic properties of this decomposition, such as entropy, mutual information and the Fisher information matrix, and show that it naturally generalizes to the more abstract context of locally compact topological groups.

1. Introduction

Lattices are discrete sets in ${\mathbb{R}}^n$ formed by all integer linear combinations of a set of independent vectors, and have found different applications, such as in information theory and communications [1,2,3]. Given a probability distribution in ${\mathbb{R}}^n$, two operations can be induced by considering the quotient of the space by a lattice: wrapping and quantization.

The wrapped distribution over the quotient is obtained by summing the probability density over each coset. It is used to define parameters for lattice coset coding, particularly for AWGN and wiretap channels, such as the flatness factor, which is, up to a constant, the $L^{\infty }$ distance from a wrapped probability distribution to a uniform one [4,5]. This factor is equivalent to the smoothing parameter, used in post-quantum lattice-based cryptography [6]. In the context of directional statistics, wrapping has been used as a standard way to construct distributions on a circle and on a torus [7].

The quantized distribution over the lattice can be defined by integrating over each fundamental domain translation, thus corresponding to the distribution of the fundamental domains after lattice-based quantization is applied. Lattice quantization has different uses in signal processing and coding: for instance, it can achieve the optimal rate-distortion trade-off and can be used for shaping in channel coding [2]. A special case of interest is when the distribution on the fundamental region is uniform, which amounts to high-resolution quantization or dithered quantization [8,9].

In this work, we relate these two operations by remarking that the random variables induced by wrapping and quantization sum up to the original one. We study information properties of this decomposition, both from classical information theory [10] and from information geometry [11], and provide some examples for the exponential and Gaussian distributions. We also propose a generalization of these ideas to locally compact groups. Probability distributions on these groups have been studied in [12], and some information-theoretic properties have been investigated in [13,14,15]. In addition to probability measures, one can also define the notions of lattice and fundamental domains on them, thereby generalizing the Euclidean case. We show that wrapping and quantization are also well defined, and provide some illustrative examples.

2. Lattices, Wrapping and Quantization

2.1. Lattices and Fundamental Domains

A lattice $\Lambda$ in ${\mathbb{R}}^n$ is a discrete additive subgroup of ${\mathbb{R}}^n$, or, equivalently, the set $\Lambda =\left\{{\alpha }_1{b}_1+\cdots +{\alpha }_k{b}_k|{\alpha }_1,\cdots ,{\alpha }_k\in \mathbb{Z}\right\}$ formed by all integer linear combinations of a set of linearly independent vector $\left\{b_1,\cdots ,b_k\right\}\subset {\mathbb{R}}^n$, called a basis of $\Lambda$. A matrix B whose column vectors form a basis is called a generator matrix of $\Lambda$, and we have $\Lambda =B{\mathbb{Z}}^k$. The lattice dimension is k, and, if $k=n$, the lattice is said to be full-rank; we henceforth consider full-rank lattices. A lattice $\Lambda$ defines an equivalence relation in ${\mathbb{R}}^n$: $x\sim y\iff x-y\in \Lambda$. The associated equivalence classes are denoted by $\overline{x}$ or $x+\Lambda$. The set of all equivalence classes is the lattice quotient ${\mathbb{R}}^n/\Lambda$, and we denote the standard projection $\pi :{\mathbb{R}}^n\to {\mathbb{R}}^n/\Lambda ,\pi \left(x\right)=\overline{x}$.

Let $\mathcal{D}$ be a Lebesgue-measurable set of ${\mathbb{R}}^n$ and $\Lambda$ a lattice. We say that $\mathcal{D}$ is a fundamental domain or a fundamental region of $\Lambda$, or that $\mathcal{D}$ tiles ${\mathbb{R}}^n$ by $\Lambda$, if (1) ${\bigcup }_{\lambda \in \Lambda }(\lambda +\mathcal{D})={\mathbb{R}}^n$, and (2) $(\lambda +\mathcal{D})\cap (\tilde{\lambda }+\mathcal{D})=\varnothing$, for all $\lambda \ne \tilde{\lambda }$ in $\Lambda$ (it is often only asked that this intersection has Lebesgue measure zero, but we require it to be empty). Given a fundamental domain $\mathcal{D}$, each coset $\overline{x}\in {\mathbb{R}}^n/\Lambda$ has a unique representative in $\mathcal{D}$, i.e., the measurable map ${\pi |}_{\mathcal{D}}:\mathcal{D}\to {\mathbb{R}}^n/\Lambda$ is a bijection. This fact suggests using a fundamental domain to represent the quotient. Each fundamental domain contains exactly one lattice point, which may be chosen as the origin. An example of a fundamental domain is the fundamental parallelotope with respect to a basis $\{b_1,\cdots ,b_n\}$, namely $\mathcal{P}(\Lambda ):=\left\{x={\alpha }_1{b}_1+\cdots +{\alpha }_n{b}_n|{\alpha }_1,\cdots ,{\alpha }_n\in \left[0,1\right[\right\}$. Another one is the Voronoi region $\mathcal{V}(\Lambda )$ of the origin, given by the points that are closer to the origin than to any other lattice point, with an appropriate choice for ties. It is a well-known fact that every fundamental domain has the same volume, denoted by $covol\Lambda :=vol\mathcal{D}=\left|detB\right|$, for any generator matrix B of $\Lambda$.

2.2. Wrapping and Quantization

Consider ${\mathbb{R}}^n$ with the Lebesgue measure $\mu$, and P a probability measure such that $P\ll \mu$. Then the probability density function (pdf) of P is $p=\frac{\mathrm{d}P}{\mathrm{d}\mu }$, the Radon–Nikodym derivative. For fixed full-rank lattice $\Lambda$ and fundamental domain $\mathcal{D}$, the wrapping of P by $\Lambda$ is the distribution $P_{\pi }:={\pi }_{*}P$ on ${\mathbb{R}}^n/\Lambda$, given by $P_{\pi }\left(A\right)=P\left({\pi }^{-1}A\right)$. For simplicity, we identify ${\mathbb{R}}^n/\Lambda$ with $\mathcal{D}$ to regard $P_{\pi }$ as a distribution over $\mathcal{D}$, and then we have $\pi :{\mathbb{R}}^n\to \mathcal{D}$ given by $(y+\lambda )\mapsto y$, for all $y\in \mathcal{D},\lambda \in \Lambda$. Using this identification, the wrapping has density $p_{\pi }=\frac{\mathrm{d}{P}_{\pi }}{\mathrm{d}\mu }$ given by

$$p_{\pi }\left(y\right)=\sum _{\lambda \in \Lambda }p(y+\lambda ).$$

(1)

A construction that is, in some sense, dual to wrapping is quantization. Note that each fundamental domain $\mathcal{D}$ partitions the space as ${\mathbb{R}}^n={⨆}_{\lambda \in \Lambda }(\lambda +\mathcal{D})$. The quantization function is the measurable map $\mathcal{Q}:{\mathbb{R}}^n\to \Lambda$, given by $(y+\lambda )\mapsto \lambda$, for $y\in \mathcal{D}$ and $\lambda \in \Lambda$. The quantized probability distribution of P on the discrete set $\Lambda$ is $P_{\mathcal{Q}}:={\mathcal{Q}}_{*}P$, given by $P_{\mathcal{Q}}\left(A\right):=P\left({\mathcal{Q}}^{-1}A\right)$. The probability mass function of the quantized distribution is then

$$p_{\mathcal{Q}}\left(\lambda \right)={\int }_{\mathcal{D}}p(y+\lambda )\mathrm{d}y.$$

(2)

Letting X be a vector random variable in ${\mathbb{R}}^n$ with distribution p, we define $X_{\pi }:=\pi \left(X\right)$ and $X_{\mathcal{Q}}:=\mathcal{Q}\left(X\right)$ the wrapped and quantized random variables, respectively. By definition, they are distributed according to $p_{\pi }$ and $p_{\mathcal{Q}}$. Interestingly, they sum up to the original one:

$$X=X_{\pi }+X_{\mathcal{Q}},$$

(3)

since $\pi +\mathcal{Q}={\mathrm{id}}_{{\mathbb{R}}^n}$. Note also that $X_{\pi }+X_{\mathcal{Q}}$ has the same distribution as $(X_{\pi },X_{\mathcal{Q}})$, by the bimeasurable bijection $y+\lambda \mapsto (y,\lambda )$. These factors, however, are not independent, since, in general, $p(y+\lambda )\ne p_{\pi }\left(y\right)p_{\mathcal{Q}}\left(\lambda \right)$. The difference between $p\left(x\right)$ and $(p_{\pi }\otimes p_{\mathcal{Q}})\left(x\right):=p_{\pi }\left(\pi \left(x\right)\right)p_{\mathcal{Q}}\left(\mathcal{Q}\left(x\right)\right)$ shall be illustrated in the following examples. Note that the expression for the quantized distribution depends on the choice of fundamental domain, while the wrapped distribution does not, up to a lattice translation.

We say a random variable X over $[0,\infty )$ is memoryless if $\overline{C}\left(t\right)=\overline{C}(t+s)/\overline{C}\left(s\right)$ for all $t,s$, where $\overline{C}\left(t\right):=P[X>t]$ is the tail distribution function. In particular, a memoryless distribution satisfies $\overline{C}(y+\lambda )=\overline{C}\left(y\right)\overline{C}\left(\lambda \right)$ for all $y\in \mathcal{D}$, $\lambda \in \Lambda$, which implies $p=p_{\pi }\otimes p_{\mathcal{Q}}$. The converse, however, is not true; for example, independence holds whenever p is constant on each region $\lambda +\mathcal{D}$, for $\lambda \in \Lambda$.

Example 1.

The exponential distribution, parametrized by $\nu >0$, is defined as

$$p\left(x\right)=\nu e^{-\nu x}{\mathbb{1}}_{\left[0,+\infty \right[}\left(x\right),$$

where ${\mathbb{1}}_A\left(x\right)$ takes value 1 if $x\in A$, and 0 otherwise. Choosing the lattice $\Lambda =\alpha \mathbb{Z}$, $\alpha \in {\mathbb{R}}_{+}$, and the fundamental domain $\mathcal{D}=\left[0,\alpha \right[$, one can write closed-form expressions for the wrapped and quantized distributions:

$$p_{\pi }\left(y\right)=\frac{\nu e^{-\nu y}}{1-e^{-\nu \alpha }},y\in \mathcal{D}\mathrm{and}{p}_{\mathcal{Q}}\left(\lambda \right)=e^{-\nu \lambda }\left(1-e^{-\nu \alpha }\right),\lambda \in \Lambda \cap {\mathbb{R}}_{+}.$$

(4)

Note that, in this special case, $p=p_{\pi }\otimes p_{\mathcal{Q}}$, as a consequence of memorylessness. The wrapped distribution with $\alpha =2\pi$, which amounts to a distribution on the unitary circle, is well studied in [16].

Example 2.

Consider the univariate Gaussian distribution

$$p\left(x\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}exp\left(-\frac{{(x-\mu )}^2}{2{\sigma }^2}\right)$$

and the lattice $\Lambda =\alpha \mathbb{Z}$, with fundamental domain $\mathcal{D}=\left[-\frac{\alpha }{2},\frac{\alpha }{2}\right[$, $\alpha \in {\mathbb{R}}_{+}$. The wrapped and quantized distributions are given respectively by

$$p_{\pi }\left(y\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}\sum _{i\in \mathbb{Z}}{e}^{-\frac{{(y-\mu +\alpha i)}^2}{2{\sigma }^2}},y\in \mathcal{D}\mathrm{and}{p}_{\mathcal{Q}}\left(\lambda \right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{\int }_{\lambda -\frac{\alpha }{2}}^{\lambda +\frac{\alpha }{2}}{e}^{-\frac{{(x-\mu )}^2}{2{\sigma }^2}}\mathrm{d}x,\lambda \in \Lambda .$$

The value $\alpha =2\pi$ for the wrapped distribution on a unitary circle is usually considered in directional statistics [7]. Figure 1 illustrates the original, wrapped, quantized and product distributions for different zero-mean Gaussian distributions. As can be seen in the figure, in this case, $p\left(x\right)\ne p_{\pi }\left(y\right)p_{\mathcal{Q}}\left(\lambda \right)$.

Figure 1. Example of zero-mean Gaussian distributions and their corresponding wrapped, quantized and product distributions, with $\Lambda =\mathbb{Z}$ and $\mathcal{D}=[-\frac{1}{2},\frac{1}{2}[$ for different variances: ${\sigma }^2=0.25$ (blue), ${\sigma }^2=1$ (orange), ${\sigma }^2=4$ (green).

A straightforward consequence of the decomposition (3) is

• $E\left[X\right]=E\left[X_{\pi }\right]+E\left[X_{\mathcal{Q}}\right]$;
• $Var\left[X\right]=Var\left[X_{\pi }\right]+Var\left[X_{\mathcal{Q}}\right]+Cov[X_{\pi },X_{\mathcal{Q}}]+Cov[X_{\mathcal{Q}},X_{\pi }]$,

where $E[·]$, $Var[·]$ and $Cov[·,·]$ denote, respectively, the expectation, the variance and the cross-covariance operators.

We note that different types of discretization have also been studied, other then integrating over a fundamental domain [17]. For instance, in [4,18,19], the discretized distribution is defined by restricting the original pdf $p\left(x\right)$ to the lattice $\Lambda$, and then normalizing:

$$D_{\Lambda ,c}\left(\lambda \right):=\frac{p(c+\lambda )}{{\sum }_{\tilde{\lambda }\in \Lambda }p(c+\tilde{\lambda })},$$

(5)

for a fixed $c\in \mathcal{D}$. This discretization is nothing other than the conditional distribution of $X_{\mathcal{Q}}$ given that $X_{\pi }=c$, expressed as $p_{\mathcal{Q}|\pi }\left(\lambda \right|c)=p(c+\lambda )/p_{\pi }\left(c\right)$. Moreover, when $p=p_{\pi }\otimes p_{\mathcal{Q}}$, such as in the exponential distribution, cf. example 1, then $D_{\Lambda ,c}\left(\lambda \right)=p_{\mathcal{Q}}\left(\lambda \right)$.

3. Information Properties

3.1. Information-Theoretic Measures

Let us consider a random variable X with distribution p and the induced wrapped and quantized ones, respectively, $X_{\pi }\sim p_{\pi }$ and $X_{\mathcal{Q}}\sim p_{\mathcal{Q}}$. The mutual information between $X_{\pi }$ and $X_{\mathcal{Q}}$ is defined as the Kullback–Leibler divergence $I(X_{\pi };X_{\mathcal{Q}}):={\mathrm{D}}_{\mathrm{KL}}\left(p\parallel p_{\pi }\otimes p_{\mathcal{Q}}\right)$, and is a measure of how non-independent the marginal distributions $p_{\pi }$ and $p_{\mathcal{Q}}$ are [10]. Using the theorem of change of variables, we have

$$\begin{array}{cc}\hfill I(X_{\pi };X_{\mathcal{Q}})& =E_X\left[log\frac{p\left(X\right)}{p_{\pi }\otimes p_{\mathcal{Q}}\left(X\right)}\right]\hfill \\ \hfill & =E_X[logp\left(X\right)]-E_X[log{p}_{\pi }\left(X_{\pi }\right)]-E_X[log{p}_{\mathcal{Q}}\left(X_{\mathcal{Q}}\right)]\hfill \\ \hfill & =E_X[logp\left(X\right)]-E_{X_{\pi }}[log{p}_{\pi }\left(X_{\pi }\right)]-E_{X_{\mathcal{Q}}}[log{p}_{\mathcal{Q}}\left(X_{\mathcal{Q}}\right)]\hfill \\ \hfill & =h\left(X_{\pi }\right)+H\left(X_{\mathcal{Q}}\right)-h\left(X\right).\hfill \end{array}$$

(6)

Note that, from this decomposition, we have $h\left(X\right)\le h\left(X_{\pi }\right)+H\left(X_{\mathcal{Q}}\right)$.

Proposition 1.

Let X be a real random variable, and $X_{\pi }$ and $X_{\mathcal{Q}}$ the respective wrapped and quantized random variables, using the lattice $\alpha \mathbb{Z}$. Denote ${\mu }_{\mathcal{Q}}:=E\left[X_{\mathcal{Q}}\right]$ and ${\sigma }_{\mathcal{Q}}^2:=Var\left[X_{\mathcal{Q}}\right]$. If X has support $[0,\infty )$, then the mutual information $I(X_{\pi };X_{\mathcal{Q}})$ between $X_{\pi }$ and $X_{\mathcal{Q}}$ is upper-bounded by

$$I(X_{\pi };X_{\mathcal{Q}})<log\left(e\left({\mu }_{\mathcal{Q}}+\alpha /2\right)\right)-h\left(X\right).$$

(7)

If X has support $\mathbb{R}$, then $I(X_{\pi };X_{\mathcal{Q}})$ is upper-bounded by

$$I(X_{\pi };X_{\mathcal{Q}})<\frac{1}{2}log\left(2\pi e{\sigma }_{\mathcal{Q}}^2\right)+\frac{2loge}{exp\left(2{\pi }^2{\alpha }^{-2}{\sigma }_{\mathcal{Q}}^2\right)-1}-h\left(X\right).$$

(8)

Proof. 

First, $h\left(X_{\pi }\right)\le log\alpha$, since the uniform distribution maximizes entropy on a bounded support. Then, note that the mean and variance of the integer-valued random variable ${\alpha }^{-1}{X}_{\mathcal{Q}}$ are ${\alpha }^{-1}{\mu }_{\mathcal{Q}}$ and ${\alpha }^{-2}{\sigma }_{\mathcal{Q}}^2$, respectively. For (7), use that, for positive integer random variables, $H\left(X_{\mathcal{Q}}\right)<log\left(e\left({\mu }_{\mathcal{Q}}/\alpha +1/2\right)\right)$, as in [20] (Theorem 8); for (8), the upper-bound for integer-valued random variables from [20] (Theorem 10) gives us $H\left(X_{\mathcal{Q}}\right)<\frac{1}{2}log\left(2\pi e{\alpha }^{-2}{\sigma }_{\mathcal{Q}}^2\right)+\frac{2loge}{exp\left(2{\pi }^2{\alpha }^{-2}{\sigma }_{\mathcal{Q}}^2\right)-1}$. Replacing the corresponding inequalities in (6) yields the desired results. □

The following lemma can be found in [2] (Appendix 3).

Lemma 1.

$h\left(X_{\pi }\right)\le h\left(X\right)$.

Proof. 

$h\left(X\right)=h\left(X_{\pi }\right)+H\left(X_{\mathcal{Q}}\right|X_{\pi })$, and $H\left(X_{\mathcal{Q}}\right|X_{\pi })\ge 0$, since it is a discrete entropy. □

Proposition 2.

Let ${\Lambda }_{\alpha }:=\alpha \Lambda$, $\alpha >0$, be a family of lattices, with fundamental domains ${\mathcal{D}}_{\alpha }:=\alpha \mathcal{D}$.

1. 

If $\mathcal{D}$ is connected, and p is continuous and Riemann-integrable, then ${lim}_{\alpha \to 0}I(X_{\pi };X_{\mathcal{Q}})=0$.

2. 

If 0 is an interior point of $\mathcal{D}$, then ${lim}_{\alpha \to +\infty }I(X_{\pi };X_{\mathcal{Q}})=0$.

Proof. 

For $\alpha \to 0$, the proof is an adaptation of [10] (Theorem 8.3.1). Since $\mathcal{D}$ is connected and p is continuous, we can use the mean value theorem: for every $\lambda \in \Lambda$ there exists an $x_{\lambda ,\alpha }\in (\lambda +{\mathcal{D}}_{\alpha })$ such that $p\left(x_{\lambda ,\alpha }\right)vol{\mathcal{D}}_{\alpha }=p_{\mathcal{Q}}\left(\lambda \right)$. Therefore, we can write $H\left(X_{\mathcal{Q}}\right)=-{\sum }_{\lambda \in {\Lambda }_{\alpha }}p\left(x_{\lambda ,\alpha }\right)log\left(p\left(x_{\lambda ,\alpha }\right)\right)vol{\mathcal{D}}_{\alpha }-log\left(vol{\mathcal{D}}_{\alpha }\right)$, using that ${\sum }_{\lambda \in {\Lambda }_{\alpha }}p\left(x_{\lambda ,\alpha }\right)vol{\mathcal{D}}_{\alpha }=1$. The first term is an n-dimensional Riemann sum, and converges to $h\left(X\right)$ when $\alpha \to 0$, while the second term becomes arbitrarily small. Therefore, $0\le I(X_{\pi };X_{\mathcal{Q}})\le H\left(X_{\mathcal{Q}}\right)+log\left(vol{\mathcal{D}}_{\alpha }\right)-h\left(X\right)\to 0$, so $I(X_{\pi };X_{\mathcal{Q}})\to 0$.

For $\alpha \to +\infty$, note that, from Lemma 1, $I(X_{\pi };X_{\mathcal{Q}})\le H\left(X_{\mathcal{Q}}\right)$. However, by choosing $\alpha$ sufficiently large, we can make $p_{\mathcal{Q}}\left(0\right)={\int }_{{\mathcal{D}}_{\alpha }}p\left(x\right)\mathrm{d}x$ arbitrarily close to 1, since 0 is in the interior of ${\mathcal{D}}_{\alpha }$. Therefore, $H\left(X_{\mathcal{Q}}\right)$ can be made arbitrarily small. □

Example 3.

In the case of the exponential distributions, as in Example 1, the distributions of $X_{\pi }$ and $X_{\mathcal{Q}}$ are independent, i.e., $p=p_{\pi }\otimes p_{\mathcal{Q}}$, therefore $I(X_{\pi };X_{\mathcal{Q}})=0$. The mutual information and the corresponding upper bound (7) are plotted in Figure 2a, as function of the parameter ν.

Figure 2. Mutual information $I(X_{\pi };X_{\mathcal{Q}})$ and its upper bound.

Example 4.

In the case of the univariate zero-mean Gaussian distributions, as in Example 2, one can use (6) to numerically compute the mutual information $I(X_{\pi };X_{\mathcal{Q}})$, as a function of the standard deviation σ, and compare it with the upper bound (8) (Figure 2b). Interestingly, $I(X_{\pi };X_{\mathcal{Q}})$ vanishes as $\sigma \to 0$ or $\sigma \to +\infty$, which is equivalent to choosing a lattice $\Lambda =\alpha \mathbb{Z}$ with $\alpha \to 0$ or $\alpha \to +\infty$, cf. Proposition 2. The mutual information attains a maximum in $\sigma \approx 0.38$, showing this is the value for which $X_{\pi }$ and $X_{\mathcal{Q}}$ are the least independent.

3.2. Fisher Information

Let $M=\left\{p_{\theta }:\theta \in \Theta \right\}$ be a family of probability densities $p_{\theta }:{\mathbb{R}}^n\to {\mathbb{R}}_{+}$ smoothly parametrized by $\theta$ in an open set $\Theta \subset {\mathbb{R}}^d$. The Fisher information matrix is defined as the positive semi-definite matrix $G\left(\theta \right)$ with coefficients $g_{ij}\left(\theta \right)=E_{p_{\theta }}\left[{\partial }_i{\ell }_{\theta }{\partial }_j{\ell }_{\theta }\right]$, where ${\ell }_{\theta }\left(x\right):=log{p}_{\theta }\left(x\right)$. When M is a manifold satisfying certain regularity conditions [11], and G is positive definite, it becomes a Riemannian manifold with the metric given by $g_{ij}\left(\theta \right)$, called a statistical manifold. Let ⪯ denote the Loewner partial order for matrices, given by $A⪯B$ if, and only if, $B-A$ is positive semi-definite. The following results justify the name information matrix given to this quantity.

Proposition 3

([11,21]). Let X be a random variable distributed according to a distribution parametrized by θ, and $G\left(\theta \right)$ its information matrix. The following hold.

1. 

Monotonicity:if $F:\mathcal{X}\to \mathcal{Y}$ is a measurable function (i.e., a statistic) and $G_F\left(\theta \right)$ is the information matrix of $F\left(X\right)$, then $G_F\left(\theta \right)⪯G\left(\theta \right)$, with equality if, and only if, F is a sufficient statistic for θ.

2. 

Additivity:if $X,Y$ are independent random variables, then the joint information matrix satisfies $G_{(X,Y)}\left(\theta \right)=G_X\left(\theta \right)+G_Y\left(\theta \right)$.

Let X be a random variable on ${\mathbb{R}}^n$, and $X_{\pi }$ and $X_Q$ its wrapped and quantized factors, respectively. We denote their respective Fisher information matrices by $G\left(\theta \right)$, $G_{\pi }\left(\theta \right)$ and $G_{\mathcal{Q}}\left(\theta \right)$. By additivity, the Fisher information of $p_{\pi }\otimes p_{\mathcal{Q}}$ is $\tilde{G}\left(\theta \right):=G_{\pi }\left(\theta \right)+G_{\mathcal{Q}}\left(\theta \right)$, and, by monotonicity, we have both $G_{\pi }\left(\theta \right)⪯G\left(\theta \right)$ and $G_{\mathcal{Q}}\left(\theta \right)⪯G\left(\theta \right)$. It follows immediately that

$$\frac{\tilde{G}\left(\theta \right)}{2}=\frac{G_{\pi }\left(\theta \right)+G_Q\left(\theta \right)}{2}⪯G\left(\theta \right).$$

(9)

Example 5.

In the family of exponential distributions, as in Example 1, the independence of $X_{\pi }$ and $X_{\mathcal{Q}}$ implies that the Fisher information matrix is additive. Indeed, for $\Lambda =\alpha \mathbb{Z}$:

$$G\left(\nu \right)=\frac{1}{{\nu }^2},G_{\pi }\left(\nu \right)=\frac{1}{{\nu }^2}+\frac{{\alpha }^2}{2(1-cosh(\alpha \nu \left)\right)},\mathit{and}{G}_{\mathcal{Q}}\left(\nu \right)=\frac{{\alpha }^2}{2(cosh(\alpha \nu )-1)}.$$

4. A Generalization to Topological Groups

A topological group is a topological space $(G,{\tau }_G)$ that is also a group with respect to some operation · called product, and such that the inverse $g^{-1}$ and product $g·h$ are continuous. As additional requisites, we ask G to be locally compact, Hausdorff and second-countable (i.e., has a countable basis) [22]. Let ${\mathcal{B}}_G$ be the the Borel $\sigma$-algebra of G. Haar’s theorem says there is a unique (up to a constant) Radon measure on G that is invariant by left translations—we will suppose a fixed normalization, and denote both the measure and integration with respect to it by dg. The group G is said to be unimodular if dg is also invariant by right translations. Since G is $\sigma$-compact, the Haar measure is $\sigma$-finite [12].

Let $\Gamma$ be a discrete subgroup of G, which is necessarily closed, since G is Hausdorff, and countable, since G is second-countable. Let us also consider the quotient space of left cosets $G/\Gamma =\left\{\overline{g}=g\Gamma |g\in G\right\},$ which has a natural projection $\pi :G\to G/\Gamma$, given by $\pi \left(g\right)=\overline{g}$. We call $\Gamma$ a lattice if the induced Haar measure on $G/\Gamma$ is finite and bi-invariant. A particular case is when the quotient $G/\Gamma$ is compact; then $\Gamma$ is said to be a uniform lattice. A cross-section is defined as a set $\mathcal{D}\subset G$ of representatives of $G/\Gamma$ such that all cosets are uniquely represented. A fundamental domain is a measurable cross-section. It can be shown that $\Gamma$ is a lattice if, and only if, it admits a fundamental domain. Furthermore, every fundamental domain has the same measure [23,24].

Let P be a probability measure on the space $(G,{\mathcal{B}}_G)$ that is absolutely continuous with respect to the Haar measure dg. By the Radon–Nikodym theorem, we can define a density function $p=\frac{\mathrm{d}P}{\mathrm{d}g}\in L^1\left(G\right)$, such that $p\ge 0$ and $P\left(A\right)={\int }_{A}p\left(g\right)\mathrm{d}g$, for all $A\in {\mathcal{B}}_G$. The original measure can be represented as $P=p\mathrm{d}g$, and we consider the family of all such densities

$$\{\mathcal{P}\left(G\right)=p\in L^1\left(G\right)|p\ge 0\mathrm{d}g\text{-}\mathrm{a}.\mathrm{e}.,\int p\mathrm{d}g=1\}.$$

Probability distributions on locally compact groups have been studied in [12], and some information-theoretic properties have been investigated in [13,14,15]. The result that allows us to consider wrapped distributions in this context is the Weil formula, taken as a particular case of [24] (Theorem 3.4.6):

Theorem 1.

For any $f\in L^1\left(G\right)$, the wrapping $f_{\pi }\in L^1(G/\Gamma )$, $f_{\pi }\left(\overline{g}\right):={\sum }_{\lambda \in \Gamma }f\left(g\lambda \right)$ is well defined $\mathrm{d}\overline{g}$-almost everywhere, belongs to $L^1(G/\Gamma )$, and

$${\int }_{G}f\left(g\right)\mathrm{d}g={\int }_{G/\Gamma }\sum _{\lambda \in \Gamma }f\left(g\lambda \right)\mathrm{d}\overline{g}.$$

(10)

As a consequence, for every probability density $p\in \mathcal{P}\left(G\right)$, we can consider its wrapping $p_{\pi }\left(\overline{g}\right)={\sum }_{\lambda \in \Gamma }p\left(g\lambda \right)$, which is $L^1(G/\Gamma )$, non-negative and is also a probability density: ${\int }_{G/\Gamma }{p}_{\pi }\mathrm{d}\overline{g}=1$. The associated probability measure over $(G/\Gamma ,{\mathcal{B}}_{G/\Gamma })$ is $P_{\pi }=p_{\pi }\mathrm{d}\overline{g}$. This notation, suggesting $P_{\pi }$ as the push-forward measure by $\pi$, is not a coincidence, since, from Theorem 1,

$$\begin{array}{cc}\hfill {\pi }_{*}P\left(A\right)& ={\int }_G{\mathbb{1}}_A\left(\pi \left(g\right)\right)p\left(g\right)\mathrm{d}g\hfill \\ \hfill & ={\int }_{G/\Gamma }\sum _{\lambda \in \Gamma }{\mathbb{1}}_A\left(\pi \left(g\right)\right)p\left(g\lambda \right)\mathrm{d}\overline{g}\hfill \\ \hfill & ={\int }_{G/\Gamma }{\mathbb{1}}_A\left(\overline{g}\right)p_{\pi }\left(\overline{g}\right)\mathrm{d}\overline{g}=P_{\pi }\left(A\right).\hfill \end{array}$$

Analogously, given a fundamental domain $\mathcal{D}$, it is possible to define a quantization map $\mathcal{Q}:G\to \Lambda$ by $\mathcal{Q}\left(g\lambda \right)=\lambda$, for every $g\in \mathcal{D},\lambda \in \Gamma$, which is unique since $G={⨆}_{g\in \mathcal{D}}g\Gamma$. The quantized probability distribution is the discrete probability measure $P_{\mathcal{Q}}$ over $\Lambda$, defined by the mass function $p_{\mathcal{Q}}\left(\lambda \right)={\int }_{\mathcal{D}}p\left(g\lambda \right)\mathrm{d}g$, or as the push-forward measure ${\mathcal{Q}}_{*}P$.

If X is distributed according to p, and $X_{\pi }=\pi \left(X\right)\sim p_{\pi }$, $X_{\mathcal{Q}}=\mathcal{Q}\left(X\right)\sim p_{\mathcal{Q}}$, then $X=X_{\pi }·X_{\mathcal{Q}}$, again, as a consequence of $g\mapsto \left(\pi \left(g\right),\mathcal{Q}\left(g\right)\right)$ being a measurable bijection whose inverse is the product $\pi \left(g\right)·\mathcal{Q}\left(g\right)$. Despite being an abstract definition, this framework expands the scope of the previous approach, cf. examples below. In the following, let $\Lambda \subset {\mathbb{R}}^n$ be a full-rank lattice, and ${\Lambda }_s\subset \Lambda$ be a full-rank sublattice, as defined in Section 2.

Example 6.

Let $G={\mathbb{R}}^n$ and $\Gamma =\Lambda$. This recovers the approach from Section 2 as a particular case.

Example 7.

Let $G=\Lambda$ and $\Gamma ={\Lambda }_s$. A fundamental domain is a choice $\mathcal{D}=\left\{d_1,\cdots ,d_k\right\}$ of $k=|\Lambda /{\Lambda }_s|$ points, where each point corresponds to a coset $\overline{\lambda }=(\lambda +{\Lambda }_s)\in \Lambda /{\Lambda }_s$. Of particular interest are Voronoi constellations [25,26] where the coset leaders are selected, with some choice made for ties. Since Λ is discrete, the Haar measure is the counting measure $\mu \left(A\right)=\left|A\right|$, and $p:\Lambda \to [0,1]$. The wrapped and quantized distributions are $p_{\pi }\left(\overline{\lambda }\right)={\sum }_{{\lambda }_s\in {\Lambda }_s}p(\lambda +{\lambda }_s)$, and $p_{\mathcal{Q}}\left({\lambda }_s\right)={\sum }_{i=1}^{k}p(d_i+{\lambda }_s)$.

Example 8.

Let $G={\mathbb{R}}^n/{\Lambda }_s$ (a torus) and $\Gamma ={\pi }_s(\Lambda )$ (the projection of Λ to G). Then ${\pi }_s(\Lambda )$ consists of a finite family of cosets ${\overline{\lambda }}_1,\cdots ,{\overline{\lambda }}_k$, for $k=|\Lambda /{\Lambda }_s|$, and a choice of fundamental domain $\overline{\mathcal{D}}$ is the projection of a fundamental domain $\mathcal{D}$ of Λ. There are some standard choices for the distribution on G, such as a wrapping from the Euclidean space and the bivariate von Mises distribution [7] (Section 11.4). Then, $p_{\pi }\left(\overline{x}\right)={\sum }_{i=1}^{k}p(\overline{x}+{\overline{\lambda }}_i)$ and $p_{\mathcal{Q}}\left({\overline{\lambda }}_i\right)={\int }_{\overline{\mathcal{D}}}p(\overline{x}+{\overline{\lambda }}_k)\mathrm{d}\overline{x}$, and, in the particular case where $p\left(\overline{x}\right)={\sum }_{{\lambda }_s\in {\Lambda }_s}p(x+{\lambda }_s)$ is a ${\Lambda }_s$-wrapped distribution, they become $p_{\pi }\left(\overline{x}\right)={\sum }_{i=1}^k{\sum }_{{\lambda }_s\in {\Lambda }_s}p(x+{\lambda }_s+{\lambda }_i)$ and $p_{\mathcal{Q}}\left({\overline{\lambda }}_i\right)={\sum }_{{\lambda }_s\in {\Lambda }_s}{\int }_{\mathcal{D}}p(x+{\lambda }_s+{\lambda }_i)\mathrm{d}x$.

Example 9.

Let $G={\mathbb{F}}_q^n$ (a finite field) or $G={\mathbb{Z}}_q^n$, and $\Gamma =\mathcal{C}$ (any linear block code). A fundamental domain can be a finite set of points that tiles the space by $\mathcal{C}$. The distributions then become finite sums, such as in Example 7.

Example 10.

Let $G=\mathrm{SL}(n,\mathbb{R})$, the Lie group of square matrices with determinant 1, and $\Gamma =\mathrm{SL}(n,\mathbb{Z})$ (the subgroup of integer matrices). This is, in fact, a lattice, since for $n=2$, $vol(G/\Gamma )=\sqrt{2}\zeta \left(2\right)$ where ζ is the Riemann zeta function, and for $n>2$ the finite covolume is calculated in [27], where descriptions of fundamental domains are also given.

5. Conclusions

In this work, we have studied the decomposition of a random variable through lattices into its wrapping and quantization terms. Generalization of examples and of Proposition 1 to higher dimensions constitutes a work in progress. We have also proposed a generalization of this decomposition to topological groups; in particular, this allows one to study information theory on such abstract spaces, which is another perspective for future work.