Arrow Contraction and Expansion in Tropical Diagrams

Abstract

Arrow contraction applied to a tropical diagram of probability spaces is a modification of the diagram, replacing one of the morphisms with an isomorphism while preserving other parts of the diagram. It is related to the rate regions introduced by Ahlswede and Körner. In a companion article, we use arrow contraction to derive information about the shape of the entropic cone. Arrow expansion is the inverse operation to the arrow contraction.

1. Introduction

In [1], we have initiated the theory of tropical probability spaces for the systematic study of information optimization problems in information theory and artificial intelligence, such as those arising in robotics [2], neuroscience [3], artificial intelligence [4], variational autoencoders [5], information decomposition [6], and causal inference [7]. In [8], we applied the techniques to derive a dimension-reduction result for the entropic cone of four random variables.

Two of the main tools used for the latter are what we call arrow contraction and arrow expansion. They are formulated for tropical commutative diagrams of probability spaces. Tropical diagrams are points in the asymptotic cone of the metric space of commutative diagrams of probability spaces endowed with the asymptotic entropy distance. Arrows in diagrams of probability spaces are (equivalence classes of) measure-preserving maps.

Arrow contraction and expansion take a commutative diagram of probability spaces as input, modify it, but preserve important properties of the diagram. The precise results are formulated as Theorems 3 and 4 in the main text. Their formulation requires language, notation, and definitions that we review in Section 2.

However, to give an idea of the results in this paper, we now present two examples. For basic terminology and notations used in these examples below, the reader unfamiliar with them is referred either to Section 2 of the present article or in the introductory material in the article [9].

1.1. Two Examples

1.1.1. Arrow Contraction and Expansion in a Two-Fan

Suppose we are given a fan $\mathcal{Z}=\left(X\stackrel{}{\leftarrow }Z\stackrel{}{\to }Y\right)$, and we would like to complete it to a diamond

(1)

such that the entropy of V, denoted by $\left[V\right]$, equals the mutual information $[X:Y]$ between X and Y, i.e., we would like to realize the mutual information between X and Y by a pair of reductions $X\stackrel{}{\to }V$ and $Y\stackrel{}{\to }V$. This is not always possible, not even approximately. The Gacs-Körner Theorem [10] describes when such exact realization of mutual information is possible.

Arrow contraction instead produces another fan ${\mathcal{Z}}^{\prime }=\left(X\stackrel{}{\leftarrow }{Z}^{\prime }\stackrel{}{\to }V\right)$, such that the reduction $Z^{\prime }\stackrel{}{\to }X$ is an isomorphism and the relative entropy $\left[X\right|V]$ of X given V equals $\left[X\right|Y]$. By collapsing this reduction, we obtain as a diagram just the reduction $X\stackrel{}{\to }V$. If necessary, we can keep the original spaces Z and Y in the modified diagram obtaining the “broken diamond” diagram

such that $\left[V\right]=[X:Y]$. Of course, no special technique is necessary to achieve this result since it is easy to find a reduction from a tropical space $\left[X\right]$ to another tropical probability space with the prespecified entropy, as long as the Shannon inequalities are not violated.

However, a similar operation becomes non-trivial and in fact impossible without passing to the tropical limit, if instead of a single space X, there is a more complex sub-diagram as in the example in the next subsection.

To explain how arrow expansion works, we start with the chain of reductions $Z\stackrel{}{\to }X\stackrel{}{\to }V$. Can we extend it to a diamond, as in (1), so that $[X:Y|V]=0$? This is again not possible, in general. However, if we pass to tropical diagrams, then such an extension always exists.

1.1.2. One More Example of Arrow Expansion and Contraction

Consider a diagram presented in Figure 1. Such a diagram is called a ${\mathsf{\Lambda }}_3$-diagram. We would like to find a reduction $X\stackrel{}{\to }V$ so that $\left[\mathcal{X}\right|U]=[\mathcal{X}\left|V\right]$. It is not possible to achieve this within the realm of diagrams of classical probability spaces. But once we pass to the tropical limit, the reduction $\left[X\right]\stackrel{}{\to }\left[V\right]$ can be found by contracting and then collapsing the arrow $\left[Z\right]\stackrel{}{\to }\left[X\right]$, as shown in Figure 1.

Figure 1. Arrow contraction and expansion in a ${\mathsf{\Lambda }}_3$-diagram. The fan $\left(\left[X\right]\stackrel{}{\leftarrow }\left[Z\right]\stackrel{}{\to }\left[U\right]\right)$ (shown in red in the Figure) is admissible. Spaces $\left[Z_1\right]$, $\left[Z_2\right]$ and $\left[Z\right]$ belong to the co-ideal $⌊U⌋$. After the operation the part of the diagram shown in blue in the Figure is left unmodified.

Arrow contraction is closely related to the Shannon channel coding theorem. This is perhaps most obvious from the proof. Furthermore, arrow contraction has connections with rate regions, as introduced by Ahlswede and Körner, see [11,12]. These results by Ahlswede and Körner were applied by [13], resulting in a new non-Shannon information inequality. Moreover, in [13], a new proof was given of the results; this new proof is similar to the proof of the arrow contraction result in the present paper.

The main contribution of our work lies in the fact that we prove a much stronger preservation of properties of the diagram under arrow contraction.

2. Preliminaries

2.1. Probability Spaces and Their Diagrams

Our main objects of study will be commutative diagrams of probability spaces. A finite probability space X is a set with a probability measure on it, supported on a finite set. We denote by $\left|X\right|$ the cardinality of the support of the measure. The statement $x\in X$ means that point x is an atom with positive weight in X. For details see [1,9,14].

Examples of commutative diagrams of probability spaces are shown in Figure 2. The objects in such diagrams are finite probability spaces and morphisms are equivalence classes of measure-preserving maps. Two such maps are considered to be equivalent if they coincide on a set of full measurements. To record the combinatorial structure of a commutative diagram, i.e., the arrangement of spaces and morphisms, we use indexing categories, which are finite poset categories satisfying an additional property, which we describe below.

Figure 2. Examples of diagrams of probability spaces.

2.1.1. Indexing Categories

A poset category is a finite category such that there is at most one morphism between any two objects either way.

For a pair of objects $k,l$ in a poset category $\mathbf{G}=\left\{i;{\gamma }_{ij}\right\}$, such that there is a morphism ${\gamma }_{kl}$ in $\mathbf{G}$, we call k an ancestor of l and l a descendant of k. The set of all ancestors of an object k together with all the morphisms between them is itself a poset category and will be called a co-ideal generated by k and denoted by $⌊k⌋$. Co-ideals are also sometimes called filters. Similarly, a poset category consisting of all descendants of $k\in \mathbf{G}$ and morphisms between them will be called an ideal generated by k and denoted $⌈k⌉$.

An indexing category $\mathbf{G}=\left\{i;{\gamma }_{ij}\right\}$ used for indexing diagrams is a poset category satisfying the following additional property: for any pair of objects $i_1,i_2\in \mathbf{G}$ the the intersection of co-ideals is also a co-ideal generated by some object $i_3\in \mathbf{G}$,

$$⌊i_1⌋\cap ⌊i_2⌋=⌊i_3⌋$$

In other words, for any pair of objects $i_1,i_2\in \mathbf{G}$ there exists a least common ancestor $i_3$, i.e., $i_3$ is an ancestor to both $i_1$ and $i_2$ and any other common ancestor is also an ancestor of $i_3$. Any indexing category is initial, i.e., there is a (necessarily unique) initial object $\widehat{\mathsf{ı}}$ in it, which is the ancestor of any other object in $\mathbf{G}$, in other words $\mathbf{G}=⌈\widehat{\mathsf{ı}}⌉$.

A fan in a category is a pair of morphisms with the same domain. Such a diagram is also called a span in some literature on Category Theory. A fan $\left(i\stackrel{}{\leftarrow }k\stackrel{}{\to }j\right)$ is called minimal, if for any other fan $\left(i\stackrel{}{\leftarrow }l\stackrel{}{\to }j\right)$ included in a commutative diagram

the vertical morphism $\left(k\stackrel{}{\to }l\right)$ must be an isomorphism. Any indexing category also satisfies the property that, for any pair of objects in it, there exists a unique minimal fan with target objects of the given ones.

This terminology will also be applied to diagrams of probability spaces indexed by $\mathbf{G}$. Thus, given a space X in a $\mathbf{G}$-diagram, we can talk about its ancestors, descendants, co-ideal $⌊X⌋$, and ideal $⌈X⌉$. We use square brackets to denote tropical diagrams and spaces in them. For the (co-)ideals in tropical diagrams, in order to unclutter notations, we will write

$$⌊X⌋:=⌊\left[X\right]⌋\mathrm{and}⌈X⌉:=⌈\left[X\right]⌉$$

2.1.2. Diagrams

For an indexing category $\mathbf{G}=\left\{i;{\gamma }_{ij}\right\}$ and a category $\mathbf{Cat}$, a commutative $\mathbf{G}$-diagram $\mathcal{X}=\left\{X_i;{\chi }_{ij}\right\}$ is a functor $\mathcal{X}:\mathbf{G}\stackrel{}{\to }\mathbf{Cat}$. A diagram $\mathcal{X}$ is called minimal if it maps minimal fans in $\mathbf{G}$ to minimal fans in $\mathbf{Cat}$.

A constant $\mathbf{G}$-diagram denoted $X^{\mathbf{G}}$ is a diagram where all the objects equal to X, and all morphisms are identities.

Important examples of indexing categories are a two-fan, a diamond category, a full category ${\mathsf{\Lambda }}_n$ on n spaces, chains ${\mathbf{C}}_n$. For detailed descriptions and more examples, the reader is referred to the articles cited at the beginning of this section.

2.2. Tropical Diagrams

2.2.1. Intrinsic Entropy Distance

For a fixed indexing category $\mathbf{G}$, the space of commutative $\mathbf{G}$-diagrams will be denoted by $\mathbf{Prob}⟨\mathbf{G}⟩$. Evaluating entropy on every space in a $\mathbf{G}$ diagram gives a map

$${\mathsf{Ent}}_{*}:\mathbf{Prob}⟨\mathbf{G}⟩\stackrel{}{\to }{\mathbb{R}}^{\mathbf{G}}$$

where the target space ${\mathbb{R}}^{\mathbf{G}}$ is the space of real-valued functions on objects of $\mathbf{G}$. We endow this space with the ${\ell }^1$-norm. For a fan $\mathcal{F}=\left(\mathcal{X}\stackrel{}{\leftarrow }\mathcal{Z}\stackrel{}{\to }\mathcal{Y}\right)$ of $\mathbf{G}$-diagrams we define the entropy distance between its terminal objects by

$$\mathrm{kd}\left(\mathcal{F}\right):={∥{\mathsf{Ent}}_{*}\mathcal{Z}-{\mathsf{Ent}}_{*}\mathcal{X}∥}_1+{∥{\mathsf{Ent}}_{*}\mathcal{Z}-{\mathsf{Ent}}_{*}\mathcal{Y}∥}_1$$

and the intrinsic entropy distance between two arbitrary $\mathbf{G}$-diagrams by

$$\mathbf{k}(\mathcal{X},\mathcal{Y}):=inf\left\{\mathrm{kd}\left(\mathcal{F}\right):\mathcal{F}=\left(\mathcal{X}\stackrel{}{\leftarrow }\mathcal{Z}\stackrel{}{\to }\mathcal{Y}\right)\right\}$$

This intrinsic version of the entropy distance was introduced in [15,16]. The triangle inequality for $\mathbf{k}$ and various other properties are discussed in [1].

In the same article, a useful estimate for the intrinsic entropy distance called the Slicing Lemma is also proven. The following corollary ([1], Corollary 3.10(1)) of the Slicing Lemma will be used in the next section.

Proposition 1. 

Let $\mathbf{G}$ be an indexing category, $\mathcal{X},\mathcal{Y}\in \mathbf{Prob}⟨\mathbf{G}⟩$ and $U\in \mathbf{Prob}$ included in a pair of two fans

Then

$$\mathbf{k}(\mathcal{X},\mathcal{Y})\le {\int }_U\mathbf{k}\left(\mathcal{X}\right|u,\mathcal{Y}\left|u\right)d{p}_U\left(u\right)+2·\left[\left[\mathbf{G}\right]\right]·\mathsf{Ent}\left(U\right)$$

2.2.2. Tropical Diagrams

Points in the asymptotic cone of $(\mathbf{Prob}⟨\mathbf{G}⟩,\mathbf{k})$ are called tropical $\mathbf{G}$-diagrams and the space of all tropical $\mathbf{G}$-diagrams, denoted $\mathbf{Prob}\left[\mathbf{G}\right]$, is endowed with the asymptotic entropy distance. We explain this now in more detail, and a more extensive description can be found in [14].

To describe points in $\mathbf{Prob}\left[\mathbf{G}\right]$ we consider quasi-linear sequences $\overline{\mathcal{X}}:=(\mathcal{X}\left(n\right):n\in \mathbb{N})$ of $\mathbf{G}$-diagrams. That is, we fix a “slowly growing” increasing function $\phi :{\mathbb{R}}_{\ge 0}\stackrel{}{\to }\mathbb{R}$ satisfying

$$t·{\int }_t^{\infty }\frac{\phi \left(t\right)}{t^2}dt\le D_{\phi }·\phi \left(t\right)$$

for some constant $D_{\phi }>0$ and any $t>1$. We call a sequence $\overline{\mathcal{X}}:=(\mathcal{X}\left(n\right):n\in \mathbb{N})$ $\phi$-quasi-linear if it satisfies the bound for all $m,n\in \mathbb{N}$

$$\mathbf{\kappa }\left(\mathcal{X}(n+m),\mathcal{X}\left(n\right)\otimes \mathcal{X}\left(m\right)\right)\le C·\phi (n+m)$$

We have shown in [14] that the space $\mathbf{Prob}\left[G\right]$ does not depend on the choice of function $\phi$ as long as it is not zero. The space of all such sequences is endowed with the asymptotic entropy distance defined by

$$\mathbf{\kappa }(\overline{\mathcal{X}},\overline{\mathcal{Y}}):=\underset{n\stackrel{}{\to }\infty }{lim}\frac{1}{n}\mathbf{k}\left(\mathcal{X}\left(n\right),\mathcal{Y}\left(n\right)\right)$$

A tropical diagram $\left[\mathcal{X}\right]$ is defined to be an equivalence class of such sequences, where two sequences $\overline{\mathcal{X}}$ and $\overline{\mathcal{Y}}$ are equivalent if $\mathbf{\kappa }(\overline{\mathcal{X}},\overline{\mathcal{Y}})=0$. The space $\mathbf{Prob}\left[\mathbf{G}\right]$ carries the asymptotic entropy distance and has the structure of a ${\mathbb{R}}_{\ge 0}$-semi-module—one can take linear combinations with non-negative coefficients of tropical diagrams. The linear entropy functional ${\mathsf{Ent}}_{*}:\mathbf{Prob}\left[\mathbf{G}\right]\stackrel{}{\to }{\mathbb{R}}^{\mathbf{G}}$ is defined by

$${\mathsf{Ent}}_{*}\left[\mathcal{X}\right]:=\underset{n\stackrel{}{\to }\infty }{lim}\frac{1}{n}{\mathsf{Ent}}_{*}\mathcal{X}\left(n\right)$$

A detailed discussion about tropical diagrams can be found in [14]. In the cited article, we show that the space $\mathbf{Prob}\left[\mathbf{G}\right]$ is metrically complete and isometrically isomorphic to a closed convex cone in some Banach space.

For $\mathbf{G}={\mathbf{C}}_k$ a chain category, containing k objects $\left\{1,\dots ,k\right\}$ and unique morphism $i\stackrel{}{\to }j$ for every pair $i\ge j$, we have shown in [14] that the space $\mathbf{Prob}\left[{\mathbf{C}}_k\right]$ is isomorphic to the following cone in $({\mathbb{R}}^k,{∥·∥}_1)$

$$\mathbf{Prob}\left[{\mathbf{C}}_k\right]\cong \left\{\left(\begin{array}{c}{x}_1\\ ⋮\\ x_k\end{array}\right):0\le x_1\le \cdots \le x_k\right\}$$

The isomorphism is given by the entropy function. Thus, we can identify tropical probability spaces (elements in $\mathbf{Prob}\left[{\mathbf{C}}_1\right]$) with non-negative numbers via entropy. We will simply write $\left[X\right]$ to mean the entropy of the space $\left[X\right]$. Along these lines, we also adopt the notations $\left[X\right|Y]$, $[X:Y]$ and $[X:Y|Z]$ for the conditional entropy and mutual information for the tropical spaces included in some diagrams.

2.3. Asymptotic Equipartition Property for Diagrams

2.3.1. Homogeneous Diagrams

A $\mathbf{G}$-diagram $\mathcal{X}$ is called homogeneous if the automorphism group $\mathrm{Aut}\left(\mathcal{X}\right)$ acts transitively on every space in $\mathcal{X}$. Homogeneous probability spaces are uniform. For more complex indexing categories, this simple description is not sufficient.

2.3.2. Tropical Homogeneous Diagrams

The subcategory of all homogeneous $\mathbf{G}$-diagrams will be denoted $\mathbf{Prob}{⟨\mathbf{G}⟩}_{\mathsf{h}}$ and we write $\mathbf{Prob}{⟨\mathbf{G}⟩}_{\mathsf{h},\mathsf{m}}$ for the category of minimal homogeneous $\mathbf{G}$-diagrams. These spaces are invariant under the tensor product. Thus, they are metric Abelian monoids.

Passing to the tropical limit, we obtain spaces of tropical (minimal) homogeneous diagrams that we denote $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$ and $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h},\mathsf{m}}$.

2.3.3. Asymptotic Equipartition Property

In [1] the following theorem is proven

Theorem 1. 

Suppose $\mathcal{X}\in \mathbf{Prob}⟨\mathbf{G}⟩$ is a $\mathbf{G}$-diagram of probability spaces for some fixed indexing category $\mathbf{G}$. Then, there exists a sequence $\overline{\mathcal{H}}={\left({\mathcal{H}}_n\right)}_{n=0}^{\infty }$ of homogeneous $\mathbf{G}$-diagrams such that

$$\frac{1}{n}\mathbf{k}({\mathcal{X}}^n,{\mathcal{H}}_n)\le C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])·\sqrt{\frac{{ln}^{3}n}{n}}$$

(2)

where $C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])$ is a constant only depending on $|X_0|$ and $\left[\right[\mathbf{G}\left]\right]$.

The approximating sequence of homogeneous diagrams is evidently quasi-linear with the defect bounded by the admissible function

$$\phi \left(t\right):=2C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])·t^{3/4}\ge 2C\left(\right|X_0|,\left[\left[\mathbf{G}\right]\right])·t^{1/2}·{ln}^{3/2}t$$

Thus, Theorem 1 above states that $\mathsf{L}\left(\mathbf{Prob}⟨\mathbf{G}⟩\right)\subset \mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$. On the other hand, we have shown in [14] that the space of linear sequences $\mathsf{L}\left(\mathbf{Prob}⟨\mathbf{G}⟩\right)$ is dense in $\mathbf{Prob}\left[\mathbf{G}\right]$. Combining the two statements, we obtain the following theorem.

Theorem 2. 

For any indexing category $\mathbf{G}$, the space $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$ is dense in $\mathbf{Prob}\left[\mathbf{G}\right]$. Similarly, the space $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h},\mathsf{m}}$ is dense in $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{m}}$.

It is possible that the spaces $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$ and $\mathbf{Prob}\left[\mathbf{G}\right]$ coincide. At this time, we have neither a proof nor a counterexample to this conjecture.

2.4. Conditioning in Tropical Diagrams

For a tropical $\mathbf{G}$-diagram $\left[\mathcal{X}\right]$ containing a space $\left[U\right]$ we defined a conditioned diagram $\left[\mathcal{X}\right|U]$. It can be understood as the tropical limit of the sequence $\left(\mathcal{X}\left(n\right)\right|u_n)$, where $\left(\mathcal{X}\right(n\left)\right)$ is the homogeneous approximation of $\left[\mathcal{X}\right]$, $U\left(n\right)$ is the space in $\mathcal{X}\left(n\right)$ that corresponds to $\left[U\right]$ under combinatorial isomorphism and $u_n$ is any atom in $U\left(n\right)$.

We have shown in [9] that operation of conditioning is Lipschitz-continuous with respect to the asymptotic entropy distance.

3. Arrow Contraction

3.1. Arrow Collapse, Arrow Contraction, and Arrow Expansion

3.1.1. Prime Morphisms

A morphism ${\gamma }_{ij}:i\stackrel{}{\to }j$ in an indexing category $\mathbf{G}=\left\{i;{\gamma }_{ij}\right\}$ will be called prime if it cannot be factored into a composition of two non-identity morphisms in $\mathbf{G}$. A morphism in a $\mathbf{G}$-diagram indexed by a prime morphism in $\mathbf{G}$ will also be called prime.

3.1.2. Arrow Collapse

Suppose $\mathcal{Z}$ is a $\mathbf{G}$-diagram such that for some pair $i,j\in \mathbf{G}$, the prime morphism ${\zeta }_{ij}:Z_i\stackrel{}{\to }{Z}_j$ is an isomorphism. Arrow collapse applied to $\mathcal{Z}$ results in a new diagram ${\mathcal{Z}}^{\prime }$ obtained from $\mathcal{Z}$ by identifying $Z_i$ and $Z_j$ via the isomorphism ${\zeta }_{ij}$. The combinatorial type of ${\mathcal{Z}}^{\prime }$ is different from that of $\mathcal{Z}$. The spaces $Z_i$ and $Z_j$ are replaced by a single space, and the new space will inherit all the morphisms in $\mathcal{Z}$ with targets and domains $Z_i$ and $Z_j$.

3.1.3. Arrow Contraction and Expansion

Arrow contraction and expansion are two operations on tropical $\mathbf{G}$-diagrams. Roughly speaking, arrow contraction applied to a tropical $\mathbf{G}$-diagram $\left[\mathcal{Z}\right]$ results in another tropical $\mathbf{G}$-diagram $\left[{\mathcal{Z}}^{\prime }\right]$ such that one of the arrows becomes an isomorphism, while some parts of the diagram are not modified. Arrow expansion is an inverse operation to arrow contraction.

3.1.4. Admissible and Reduced Sub-Fans

An admissible fan in a $\mathbf{G}$-diagram $\mathcal{Z}$ is a minimal fan $X\stackrel{}{\leftarrow }Z\stackrel{}{\to }U$, such that Z is the initial space of $\mathcal{Z}$ and any space in $\mathcal{Z}$ belongs either to the co-ideal $⌈X⌉$ or ideal $⌊U⌋$. For example, in the left-most diagram of Figure 1, the fan $X\stackrel{}{\leftarrow }Z\stackrel{}{\to }U$ is admissible, while $X_1\stackrel{}{\leftarrow }{Z}_1\stackrel{}{\to }U$ or $X\stackrel{}{\leftarrow }Z\stackrel{}{\to }{Z}_2$ are not.

An admissible fan $X\stackrel{}{\leftarrow }Z\stackrel{}{\to }U$ in a diagram will be called reduced if the morphism $Z\stackrel{}{\to }X$ is an isomorphism.

3.2. The Contraction Theorem

Our aim is to prove the following theorem.

Theorem 3. 

Let $\left(\left[X\right]\stackrel{}{\leftarrow }\left[Z\right]\stackrel{}{\to }\left[U\right]\right)$ be an admissible fan in some tropical $\mathbf{G}$-diagram $\left[\mathcal{Z}\right]$. Then for every $\epsilon >0$ there exists a $\mathbf{G}$-diagram $\left[{\mathcal{Z}}^{\prime }\right]$ containing an admissible fan $\left(\left[X^{\prime }\right]\stackrel{}{\leftarrow }\left[Z^{\prime }\right]\stackrel{}{\to }\left[U^{\prime }\right]\right)$, corresponding to the original admissible fan through the combinatorial isomorphism, such that, with the notations $\mathcal{X}=⌈X⌉$ and ${\mathcal{X}}^{\prime }=⌈X^{\prime }⌉$, the diagram $\left[{\mathcal{Z}}^{\prime }\right]$ satisfies

(i) 

$\mathbf{\kappa }(\left[{\mathcal{X}}^{\prime }\right|U^{\prime }],\left[\mathcal{X}\right|U])\le \epsilon$

(ii) 

$\mathbf{\kappa }({\mathcal{X}}^{\prime },\mathcal{X})\le \epsilon$

(iii) 

$\left[Z^{\prime }\right|X^{\prime }]\le \epsilon$

It is not clear that constructing diagrams ${\mathcal{Z}}^{\prime }$ as in the theorem above for a sequence of values of parameter $\epsilon$ decreasing to 0, we can obtain a convergent sequence in $\mathbf{Prob}\left[\mathbf{G}\right]$ with the limiting diagram satisfying conclusions of the theorem with $\epsilon =0$. If $\mathbf{Prob}\left[\mathbf{G}\right]$ were a locally compact space, which is an open question at the moment. The convergence would be guaranteed, and then $\epsilon$ in the theorem above could be replaced by 0.

The proof of Theorem 3 is based on the following proposition, which will be proven in Section 5.

Proposition 2. 

Let $\left(X_0\stackrel{}{\leftarrow }{Z}_0\stackrel{}{\to }U\right)$ be an admissible fan in some homogeneous $\mathbf{G}$-diagram of probability spaces $\mathcal{Z}$. Then there exists a $\mathbf{G}$-diagram ${\mathcal{Z}}^{\prime }$ containing the admissible fan $\left(X_0^{\prime }\stackrel{}{\leftarrow }{Z}_0^{\prime }\stackrel{}{\to }{U}^{\prime }\right)$ such that, with the notations $\mathcal{X}:=⌈X_0⌉$ and ${\mathcal{X}}^{\prime }:=⌈X_0^{\prime }⌉$, it holds that

(1) 

$\mathcal{X}|u={\mathcal{X}}^{\prime }|u^{\prime }$ for any $u\in U$ and $u^{\prime }\in U^{\prime }$.

(2) 

$\mathbf{\kappa }(\mathcal{X},{\mathcal{X}}^{\prime })\le \mathbf{k}(\mathcal{X},{\mathcal{X}}^{\prime })\le 20·\left[\left[\mathbf{G}\right]\right]$

(3) 

$\left[Z_0^{\prime }\right|X_0^{\prime }]\le 4lnln\left|X_0\right|$

Proof of Theorem 3. 

First, we assume that $\left[\mathcal{Z}\right]$ is a homogeneous tropical diagram. It means that it can be represented by a quasi-linear sequence ${\left(\mathcal{Z}\left(n\right)\right)}_{n\in {\mathbb{N}}_0}$ of homogeneous diagrams, with defect of the sequence bounded by the function $\phi \left(t\right):=C·t^{3/4}$ for some $C\ge 0$. This means that for any $m,n\in \mathbb{N}$

$$\begin{array}{cc}\hfill \mathbf{\kappa }\left(\mathcal{Z}\right(m)\otimes \mathcal{Z}(n),\mathcal{Z}(m+n\left)\right)& \le \phi (m+n)\hfill \\ \hfill \mathbf{\kappa }({\mathcal{Z}}^m\left(n\right),\mathcal{Z}(m·n))& \le D_{\phi }·m·\phi \left(n\right)\hfill \end{array}$$

where $D_{\phi }$ is some constant depending on $\phi$, see [14].

Fix a number $n\in \mathbb{N}$ and apply Proposition 2 to the homogeneous diagram $\mathcal{Z}\left(n\right)$, containing the admissible fan $X_0\left(n\right)\stackrel{}{\leftarrow }{Z}_0\left(n\right)\stackrel{}{\to }U\left(n\right)$ and sub-diagram $\mathcal{X}\left(n\right)=⌈X_0\left(n\right)⌉$. As a result, we obtain a diagram ${\mathcal{Z}}^{″}$ containing the fan $X_0^{″}\stackrel{}{\leftarrow }{Z}_0^{″}\stackrel{}{\to }{U}^{″}$ and the sub-diagram ${\mathcal{X}}^{″}=⌈X_0^{″}⌉$, such that

$$\begin{array}{cc}{\mathcal{X}}^{″}|u^{″}=\mathcal{X}\left(n\right)|u\hfill & \hfill \mathrm{for}\mathrm{any}{u}^{″}\in U^{″}\mathrm{and}u\in U\left(n\right)\\ \mathbf{\kappa }({\mathcal{X}}^{″},\mathcal{X}\left(n\right))\le 20\left[\left[\mathbf{G}\right]\right]\hfill & \hfill \\ \left[Z_0^{″}\right|X_0^{″}]\le 4lnln\left|X_0\left(n\right)\right|\hfill & \hfill \end{array}$$

(3)

Recall that for a diagram $\mathcal{A}$ of probability spaces, we denote by $\overrightarrow{\mathcal{A}}$ the tropical diagram represented by the linear sequence $({\mathcal{A}}^k:k\in {\mathbb{N}}_0)$. As an element of a closed convex cone $\mathbf{Prob}\left[\mathbf{G}\right]$, it can be scaled by an arbitrary non-negative real number; see, for instance, Section 2.3.5 in [14]. For example, $\frac{1}{n}\overrightarrow{\mathcal{A}}$ is represented by the sequence $({\mathcal{A}}^{⌊\frac{k}{n}⌋}:k\in {\mathbb{N}}_0)$.

Define the two tropical diagrams

$$\begin{array}{cc}\hfill \left[{\mathcal{Z}}^{\prime }\right]& :=\frac{1}{n}\overrightarrow{{\mathcal{Z}}^{″}}\hfill \\ \hfill [\tilde{\mathcal{Z}}]& :=\frac{1}{n}\overrightarrow{\mathcal{Z}\left(n\right)}\hfill \end{array}$$

Since ${\mathcal{X}}^{″}|u^{″}$ does not depend on $u^{″}$ and $\mathcal{X}\left(n\right)|u$ does not depend on u we have $\left[{\mathcal{X}}^{\prime }\right|U^{\prime }]=(1/n)·\overrightarrow{\left({\mathcal{X}}^{″}\right|u^{″})}$ and $\left[\tilde{\mathcal{X}}\right|\tilde{U}]=(1/n)·\overrightarrow{\left(\mathcal{X}\right(n\left)\right|u)}$. From (3), we obtain

$$\begin{array}{cc}\hfill & \left[{\mathcal{X}}^{\prime }\right|U^{\prime }]=\left[\tilde{\mathcal{X}}\right|\tilde{U}]\hfill \\ \hfill & \mathbf{\kappa }\left(\left[{\mathcal{X}}^{\prime }\right],\left[\tilde{\mathcal{X}}\right]\right)\le \frac{20\left[\right[\mathbf{G}\left]\right]}{n}\hfill \\ \hfill & \left[Z_0^{\prime }\right|X_0^{\prime }]\le \frac{4lnln|X_0\left(n\right)|}{n}\hfill \end{array}$$

(4)

The distance between $\left[\tilde{\mathcal{Z}}\right]$ and $\left[\mathcal{Z}\right]$ can be bounded as follows

$$\begin{array}{cc}\hfill \mathbf{\kappa }\left(\left[\tilde{\mathcal{Z}}\right],\left[\mathcal{Z}\right]\right)& =\frac{1}{n}\mathbf{\kappa }\left(\overrightarrow{\mathcal{Z}\left(n\right)},n·\left[\mathcal{Z}\right]\right)=\frac{1}{n}\underset{m\stackrel{}{\to }\infty }{lim}\frac{1}{m}\mathbf{\kappa }\left({\mathcal{Z}}^m\left(n\right),\mathcal{Z}(m·n)\right)\hfill \\ \hfill & \le \frac{1}{n}{D}_{\phi }·\phi \left(n\right)\hfill \end{array}$$

(5)

This also implies

$$\mathbf{\kappa }\left(\left[\tilde{\mathcal{X}}\right],\left[\mathcal{X}\right]\right)\le \frac{1}{n}{D}_{\phi }·\phi \left(n\right)$$

(6)

Since conditioning is a Lipschitz-continuous operation with Lipschitz constant 2, we also have

$$\mathbf{\kappa }\left(\left[\tilde{\mathcal{X}}\right|\tilde{U}],\left[\mathcal{X}\right|U]\right)\le \frac{2}{n}{D}_{\phi }·\phi \left(n\right)$$

(7)

Combining the estimates in (4)–(7) we obtain

$$\begin{array}{cc}\hfill & \mathbf{\kappa }\left(\left[{\mathcal{X}}^{\prime }\right|U^{\prime }],\left[\mathcal{X}\right|U]\right)\le 2D_{\phi }·\frac{\phi \left(n\right)}{n}\hfill \\ \hfill & \mathbf{\kappa }\left(\left[{\mathcal{X}}^{\prime }\right],\left[\mathcal{X}\right]\right)\le \frac{20\left[\right[\mathbf{G}\left]\right]}{n}+D_{\phi }\frac{\phi \left(n\right)}{n}\hfill \\ \hfill & \left[Z_0^{\prime }\right|X_0^{\prime }]\le \frac{4lnln|X_0\left(n\right)|}{n}+2D_{\phi }\frac{\phi \left(n\right)}{n}\hfill \end{array}$$

Please note that $|X_0\left(n\right)|$ grows at most exponentially (it is bounded by ${\mathbf{e}}^{n(\left[X_0\right]+C)}$ for some C) and $\phi$ is a strictly sub-linear function. Thus, by choosing sufficiently large n depending on the given $\epsilon >0$, we obtain $\left[{\mathcal{Z}}^{\prime }\right]$, satisfying conclusions of the theorem for homogeneous $\left[\mathcal{Z}\right]$.

To prove the theorem in full generality, observe that all the quantities on the right-hand side of the inequalities are Lipschitz-continuous. Since $\mathbf{Prob}{\left[\mathbf{G}\right]}_{\mathsf{h}}$ is dense in $\mathbf{Prob}\left[\mathbf{G}\right]$ the theorem extends to any $\left[\mathcal{Z}\right]$ by first approximating it with any precision by a homogeneous configuration and applying the argument above. □

3.3. The Expansion Theorem

The following theorem is complementary to Theorem 3. The expansion applied to a diagram containing a reduced admissible fan produces a diagram with an admissible fan, such that the contraction of it is the original diagram. Thus, arrow expansion is a right inverse of the arrow contraction operation.

In general, contraction erases some information stored in the diagram, so there are many right inverses. We prove the theorem below by providing a simple construction of one such right inverse.

Theorem 4. 

Let $\left(\left[X\right]\stackrel{}{\leftarrow }\left[Z^{\prime }\right]\stackrel{}{\to }\left[U^{\prime }\right]\right)$ be a reduced admissible fan in some tropical $\mathbf{G}$-diagram $\left[{\mathcal{Z}}^{\prime }\right]$ and $\lambda >0$. Let $\left[\mathcal{X}\right]:=⌈X⌉$. Then there exists a $\mathbf{G}$-diagram $\left[\mathcal{Z}\right]$ containing the copy of $\left[\mathcal{X}\right]$, such that the corresponding admissible fan $\left(\left[X\right]\stackrel{}{\leftarrow }\left[Z\right]\stackrel{}{\to }\left[U\right]\right)$ has $\left[Z\right|X]=\lambda$ and $\left[\mathcal{X}\right|U]=\left[\mathcal{X}\right|U^{\prime }]$.

Proof. 

Let $\left[W\right]$ be a tropical probability space with entropy equal to $\lambda$. For any reduction of tropical spaces $\left[A\right]\stackrel{}{\to }\left[B\right]$, there are natural reductions

$$\begin{array}{cc}\hfill \left(\left[A\right]+\left[W\right]\right)& \stackrel{}{\to }(\left[B\right]+\left[W\right])\hfill \\ \hfill \left(\left[A\right]+\left[W\right]\right)& \stackrel{}{\to }\left[W\right]\hfill \end{array}$$

We construct the diagram $\left[\mathcal{Z}\right]$ by replacing every space $\left[V\right]$ in the ideal $⌊U^{\prime }⌋$ with $\left[U\right]+\left[W\right]$. Every morphism $\left[V_1\right]\stackrel{}{\to }\left[V_2\right]$ within $⌊U^{\prime }⌋$ is replaced by

$$\left(\left[V_1\right]+\left[W\right]\right)\stackrel{}{\to }(\left[V_2\right]+\left[W\right])$$

And any morphism from $\left[V\right]$ in $⌊U^{\prime }⌋$ to a space $\left[Y\right]$ in $⌈X⌉$ is replaced by a composition

$$\left(\left[V\right]+\left[W\right]\right)\stackrel{}{\to }\left[V\right]\stackrel{}{\to }\left[Y\right]$$

Clearly, the resulting diagram satisfies the conclusion of the theorem. □

The rest of the article is devoted to the development of the necessary tools and the proof of Proposition 2.

4. Local Estimate

In this section, we derive a bound, very similar to Fano’s inequality, on the intrinsic entropic distance between two diagrams of probability spaces with the same underlying diagram of sets. The bound will be in terms of the total variation distance between two distributions corresponding to the diagrams of probability spaces. It will be used in the next section to prove Proposition 2.

4.1. Distributions

4.1.1. Distributions on Sets

For a finite set S we denote by $\Delta S$ the collection of all probability distributions on S and by $\parallel {\pi }_1-{\pi }_2{\parallel }_1$ we denote the total variation distance between ${\pi }_1,{\pi }_2\in \Delta S$.

4.1.2. Distributions on Diagrams of Sets

Let $\mathbf{Set}$ denote the category of finite sets and surjective maps. For an indexing category $\mathbf{G}$, we denote by $\mathbf{Set}⟨\mathbf{G}⟩$ the category of $\mathbf{G}$-diagrams in $\mathbf{Set}$. That is, objects in $\mathbf{Set}⟨\mathbf{G}⟩$ are commutative diagrams of sets indexed by the category $\mathbf{G}$, the spaces in such a diagram are finite sets, and arrows represent surjective maps, subject to commutativity relations.

For a diagram of sets $\mathcal{S}=\left\{S_i;{\sigma }_{ij}\right\}$ we define the space of distributions on the diagram $\mathcal{S}$ by

$$\Delta \mathcal{S}:=\left\{\left({\pi }_i\right)\in \prod _i\Delta S_i:{\left({\sigma }_{ij}\right)}_{*}{\pi }_i={\pi }_j\right\}$$

where $f_{*}:\Delta S\stackrel{}{\to }\Delta S^{\prime }$ is the affine map induced by a surjective map $f:S\stackrel{}{\to }{S}^{\prime }$. If $S_0$ is the initial space of $\mathcal{S}$, then there is an isomorphism

$$\begin{array}{cc}\hfill \Delta S_0& \stackrel{\cong }{\leftrightarrow }\Delta \mathcal{S}\hfill \\ \hfill \Delta S_0\ni {\pi }_0& \mapsto \left\{{\left({\sigma }_{0i}\right)}_{*}{\pi }_0\right\}\in \Delta \mathcal{S}\hfill \\ \hfill \Delta S_0\ni {\pi }_0& ↤\left\{{\pi }_i\right\}\in \Delta \hfill \end{array}$$

(8)

Using the isomorphism (8) we define total variation distance between two distributions $\pi ,{\pi }^{\prime }\in \Delta \mathcal{S}$ as

$${∥\pi -{\pi }^{\prime }∥}_1:={∥{\pi }_0-{\pi }_0^{\prime }∥}_1$$

Given a $\mathbf{G}$-diagram of sets $\mathcal{S}=\left\{S_i;{\sigma }_{ij}\right\}$ and an element $\pi \in \Delta \mathcal{S}$ we can construct a $\mathbf{G}$-diagram of probability spaces $(\mathcal{S},\pi ):=\left\{(S_i,{\pi }_i);{\sigma }_{ij}\right\}$.

Below, we give the estimate of the entropy distance between two $\mathbf{G}$-diagrams of probability spaces $(\mathcal{S},\pi )$ and $(\mathcal{S},{\pi }^{\prime })$ in terms of the total variation distance $∥\pi -{\pi }^{\prime }∥$ between distributions.

4.2. The Estimate

The upper bound on the entropy distance, which we derive below, has two summands. One is linear in the total variation distance with the slope proportional to the log-cardinality of $S_0$. The second one is super-linear in the total variation distance, but it does not depend on $\mathcal{S}$. So, we have the following interesting observation: of course, the super-linear summand always dominates the linear one locally. However, as the cardinality of $\mathcal{S}$ becomes large, it is the linear summand that starts playing the main role. This will be the case when we apply the bound in the next section.

For $\alpha \in [0,1]$ consider a binary probability space with the weight of one of the atoms equal to $\alpha$

$${\mathbb{B}}_{\alpha }:=\left(\left\{\square ,■\right\};p(\square )=1-\alpha ,p(■)=\alpha \right)$$

Proposition 3. 

For an indexing category $\mathbf{G}$, consider a $\mathbf{G}$-diagram of sets $\mathcal{S}=\left\{S_i,{\sigma }_{ij}\right\}\in \mathbf{Set}⟨\mathbf{G}⟩$. Let $\pi ,{\pi }^{\prime }\in \Delta \mathcal{S}$ be two probability distributions on $\mathcal{S}$. Denote $\mathcal{X}:=(\mathcal{S},\pi )$, $\mathcal{Y}:=(\mathcal{S},{\pi }^{\prime })$ and $\alpha :=\frac{1}{2}{∥\pi -{\pi }^{\prime }∥}_1$. Then

$$\mathbf{k}(\mathcal{X},\mathcal{Y})\le 2\left[\left[\mathbf{G}\right]\right]\left(\alpha ·ln|S_0|+\mathsf{Ent}\left({\mathbb{B}}_{\alpha }\right)\right)$$

Proof. 

To prove the local estimate, we decompose both $\pi$ and ${\pi }^{\prime }$ into a convex combination of a common part $\widehat{\pi }$ and rests ${\pi }^{+}$ and ${\pi }^{\prime +}$. The coupling between the common parts gives no contribution to the distance and the worst possible estimate on the other parts is still enough to obtain the bound in the lemma, using Proposition 1.

Let $S_0$ be the initial set in the diagram $\mathcal{S}$. We will need the following obvious rough estimate of the entropy distance that holds for any $\pi ,{\pi }^{\prime }\in \Delta \mathcal{S}$:

$$\mathbf{k}(\mathcal{X},\mathcal{Y})\le 2\left[\left[\mathbf{G}\right]\right]·ln|S_0|$$

(9)

It can be obtained by taking a tensor product for the coupling between $\mathcal{X}$ and $\mathcal{Y}$.

Our goal now is to write $\pi$ and ${\pi }^{\prime }$ as the convex combination of three other distributions $\widehat{\pi }$, ${\pi }^{+}$ and ${\pi }^{\prime +}$ as in

$$\begin{array}{cc}\hfill \pi & =(1-\alpha )·\widehat{\pi }+\alpha ·{\pi }^{+}\hfill \\ \hfill {\pi }^{\prime }& =(1-\alpha )·\widehat{\pi }+\alpha ·{\pi }^{\prime +}\hfill \end{array}$$

with the smallest possible $\alpha \in [0,1]$.

We could do it the following way. Let ${\pi }_0$ and ${\pi }_0^{\prime }$ be the distributions on $S_0$ that correspond to $\pi$ and ${\pi }^{\prime }$ under isomorphisms (8). Let $\alpha :=\frac{1}{2}{∥\pi -{\pi }^{\prime }∥}_1$. If $\alpha =1$ then the proposition follows from the rough estimate (9), so from now on, we assume that $\alpha <1$. Define three probability distributions ${\widehat{\pi }}_0$, ${\pi }_0^{+}$ and ${\pi }_0^{\prime +}$ on $S_0$ by setting for every $x\in S_0$

$$\begin{array}{cc}\hfill {\widehat{\pi }}_0\left(x\right)& :=\frac{1}{1-\alpha }min\left\{{\pi }_0\left(x\right),{\pi }_0^{\prime }\left(x\right)\right\}\hfill \\ \hfill {\pi }_0^{+}& :=\frac{1}{\alpha }\left({\pi }_0-(1-\alpha ){\widehat{\pi }}_0\right)\hfill \\ \hfill {\pi }_0^{\prime +}& :=\frac{1}{\alpha }\left({\pi }_0^{\prime }-(1-\alpha ){\widehat{\pi }}_0\right)\hfill \end{array}$$

Denote by $\widehat{\pi },{\pi }^{+},{\pi }^{\prime +}\in \Delta \mathcal{S}$ the distributions corresponding to ${\widehat{\pi }}_0,{\pi }_0^{+},{\pi }_0^{\prime +}\in \Delta S_0$ under isomorphism (8). Thus, we have

$$\begin{array}{cc}\hfill \pi & =(1-\alpha )\widehat{\pi }+\alpha ·{\pi }^{+}\hfill \\ \hfill {\pi }^{\prime }& =(1-\alpha )\widehat{\pi }+\alpha ·{\pi }^{\prime +}\hfill \end{array}$$

Now, we construct two fans of $\mathbf{G}$-diagrams

(10)

by setting

$$\begin{array}{cc}\hfill {\tilde{X}}_i& :=\left(S_i\times \underline{\mathbb{B}}{}_{\alpha };{\tilde{\pi }}_i(s,\square )=(1-\alpha ){\widehat{\pi }}_i\left(s\right),{\tilde{\pi }}_i(s,■)=\alpha ·{\pi }_i^{+}\left(s\right)\right)\hfill \\ \hfill {\tilde{Y}}_i& :=\left(S_i\times \underline{\mathbb{B}}{}_{\alpha };{\tilde{\pi }}_i^{\prime }(s,\square )=(1-\alpha ){\widehat{\pi }}_i\left(s\right),{\tilde{\pi }}_i^{\prime }(s,■)=\alpha ·{\pi }_i^{\prime +}\left(s\right)\right)\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \tilde{\mathcal{X}}& :=\left\{{\tilde{X}}_i;{\sigma }_{ij}\times \mathbf{id}\right\}\hfill \\ \hfill \tilde{\mathcal{Y}}& :=\left\{{\tilde{Y}}_i;{\sigma }_{ij}\times \mathbf{id}\right\}\hfill \end{array}$$

The reduction in the fans in (10) is given by coordinate projections. Note that the following isomorphisms hold

$$\begin{array}{cc}\hfill \mathcal{X}|\square & \cong (\mathcal{S},\widehat{\pi })\hfill \\ \hfill \mathcal{X}|■& \cong (\mathcal{S},{\pi }^{+})\hfill \\ \hfill \mathcal{Y}|\square & \cong (\mathcal{S},\widehat{\pi })\cong \mathcal{X}|\square \hfill \\ \hfill \mathcal{Y}|■& \cong (\mathcal{S},{\pi }^{\prime +})\hfill \end{array}$$

Now we apply Proposition 1 along with the rough estimate in (9) to obtain the desired inequality

$$\begin{array}{cc}\hfill \mathbf{k}(\mathcal{X},\mathcal{Y})& \le (1-\alpha )\mathbf{k}\left(\mathcal{X}\right|\square ,\mathcal{Y}|\square )+\alpha ·\mathbf{k}\left(\mathcal{X}\right|■,\mathcal{Y}|■)\hfill \\ \hfill & +\sum _i\left[\mathsf{Ent}\left({\mathbb{B}}_{\alpha }\right|X_i)+\mathsf{Ent}\left({\mathbb{B}}_{\alpha }\right|Y_i)\right]\hfill \\ \hfill & \le 2\left[\left[\mathbf{G}\right]\right]\left(\alpha ·ln|S_0|+\mathsf{Ent}\left({\mathbb{B}}_{\alpha }\right)\right)\hfill \end{array}$$

  □

5. Proof of Proposition 2

In this section, we prove Proposition 2, which is shown below verbatim. The proof consists of the construction in Section 5.1 and estimates in Propositions 5 and 6.

5.1. The Construction

In this section, we fix an indexing category $\mathbf{G}$, a minimal $\mathbf{G}$-diagram of probability spaces $\mathcal{Z}$ with an admissible sub-fan $X_0\stackrel{}{\leftarrow }{Z}_0\stackrel{}{\to }U$. We denote $\mathcal{X}:=⌈X_0⌉$ and by $\mathbf{H}$ we denote the combinatorial type of $\mathcal{X}=\left\{X_i;{\chi }_{ij}\right\}$.

Instead of diagram $\mathcal{Z}$, we consider an extended diagram, which is a two-fan of $\mathbf{H}$-diagrams

(11)

where $\mathcal{Y}=\left\{Y_i;{\upsilon }_{ij}\right\}$ consists of those spaces in $\mathcal{Z}$, which are initial spaces of two fans with feet in U and in some space in $\mathcal{X}$. That is for every $i\in \mathbf{H}$ the space $Y_i$ is defined to be the initial space in the minimal fan $X_i\stackrel{}{\leftarrow }{Y}_i\stackrel{}{\to }U$ in $\mathcal{Z}$. It may happen that for some pair of indices $i_1,i_2\in \mathbf{H}$ the initial spaces of the fans with one feet U and the other $X_{i_1}$ and $X_{i_2}$ coincide in $\mathcal{Z}$. In $\mathcal{Y}$, however, they will be treated as separate spaces so that the combinatorial type of $\mathcal{Y}$ is $\mathbf{H}$. Starting with the diagram in (11) one can recover $\mathcal{Z}$ by collapsing all the isomorphism arrows. The initial space of $\mathcal{Y}$ will be denoted $Y_0$.

We would like to construct a new fan ${\mathcal{X}}^{\prime }\stackrel{{\pi }_1^{\prime }}{\leftarrow }{\mathcal{Y}}^{\prime }\stackrel{}{\to }{V}^{\mathbf{H}}$, such that

$$\left\{\begin{array}{cc}\mathcal{X}|u={\mathcal{X}}^{\prime }|v\hfill & \mathrm{for}\mathrm{any}u\in U\mathrm{and}v\in V\hfill \\ \mathbf{k}({\mathcal{X}}^{\prime },\mathcal{X})\le 20\left[\left[\mathbf{G}\right]\right]\hfill \\ [Y_0^{\prime }|X_0^{\prime }]\le 4lnln|X_0|\hfill \end{array}\right.$$

(12)

Once this goal is achieved, we collapse all the isomorphisms to obtain $\mathbf{G}$-diagram satisfying conditions in the conclusion of Proposition 2.

We start with a general description of the idea behind the construction, followed by a detailed argument. To introduce the new space V we take its points to be N atoms in $u_1,\dots ,u_N\in U$. Ideally, we would like to choose the atoms in such a way that $X_0|u_n$ are disjoint and cover the whole of $X_0$. It is not always possible to achieve this exactly. However, when $|X_0|$ is large, N is taken slightly larger than ${\mathbf{e}}^{[X_0:U]}$, and $u_1,\dots ,u_N$ are chosen at random, then with high probability the spaces $X_0|u_n$ will overlap only little and will cover most of $X_0$. The details of the construction follow.

We fix $N\in \mathbb{N}$ and construct several new diagrams. For each of the new diagrams, we provide a verbal and formal description.

• The space $U^N$. Points in it are independent samples of length N of points in U.
• The space $V_N=(\left\{1,\dots ,N\right\},\mathsf{unif})$. A point $n\in V_N$ should be interpreted as a choice of index in a sample $\overline{u}\in U^N$.
• The $\mathbf{H}$-diagram $\mathcal{A}$, where

  $$\begin{array}{cc}\hfill \mathcal{A}& =\left\{A_i;{\alpha }_{ij}\right\}\hfill \\ \hfill A_i& =\left(\left\{(x,n,\overline{u}):x\in X_i|u_n\right\},\mathsf{unif}\right)\hfill \\ \hfill {\alpha }_{ij}& =({\chi }_{ij},\mathrm{Id},\mathrm{Id})\hfill \end{array}$$
  A point $(x,n,\overline{u})$ in $A_i$ corresponds to the choice of a sample $\overline{u}\in U^N$, an independent choice of a member of the sample $u_n$ and a point $x\in X_i|u_n$. Recall that the original diagram $\mathcal{Z}$ was assumed to be homogeneous and, in particular, the distribution on $X_i|u_n$ is uniform. Due to the assumption on homogeneity of $\mathcal{Z}$, the space $X_i|u$ does not depend on $u\in U$. Since $V_N$ is also equipped with the uniform distribution, it follows that the distribution on $A_i$ will also be uniform.
• The $\mathbf{H}$-diagram $\mathcal{B}$, where

  $$\begin{array}{cc}\hfill \mathcal{B}& =\left\{B_i;{\beta }_{ij}\right\}\hfill \\ \hfill B_i& =\left(\left\{(x,\overline{u}):x\in \bigcup _{n=1}^N{X}_i|u_n\right\},p_{B_i}\right)\hfill \\ \hfill {\beta }_{ij}& =({\chi }_{ij},\mathrm{Id})\hfill \end{array}$$
  A point $(x,\overline{u})\in B_i$ is the choice of a sample $\overline{u}\in U^N$ and a point x in one of the fibers $X_i|u_n$, $n=1,\dots ,N$. The distribution $p_{B_i}$ on $B_i$ is chosen so that the natural projection $A_i\stackrel{}{\to }{B}_i$ is the reduction of probability spaces. Given a sample $\overline{u}$, if the fibers $X_i|u_n$ are not disjoint, then the distribution on $B_i|\overline{u}$ need not to be uniform. Below, we will give an explicit description of $p_{\mathcal{B}}$ and study the dependence of $p_{\mathcal{B}}(·|\overline{u})$ on the sample $\overline{u}\in U^N$.

These diagrams can be organized into a minimal diamond diagram of $\mathbf{H}$-diagrams, where reductions are obvious projections.

(13)

To describe the probability distribution on $\mathcal{B}$, first we define several relevant quantities:

$$\begin{array}{cc}\hfill \rho & :=\frac{|X_0\left|u\right|}{|X_0|}={\mathbf{e}}^{-[X_0:U]}\hfill \\ \hfill N(x,\overline{u})& :=\left|\left\{n\in V_N:x\in X_0|u_n\right\}\right|\hfill \\ \hfill \nu (x,\overline{u})& :=\frac{N(x,\overline{u})}{N}=p_{V_N}\left\{n\in V_N:x\in X_0|u_n\right\}\hfill \end{array}$$

(14)

Recall that the distribution $p_{\mathcal{B}}$ is completely determined by the distribution $p_{B_0}$ on the initial space of $\mathcal{B}$ via isomorphism (8). From homogeneity of $\mathcal{Z}$ it follows that distributions on both $A_0$ and ${A|}_{\overline{u}}$ are uniform. Therefore

$$p_{B_0}\left(x\right|\overline{u}):=\frac{\nu (x,\overline{u})}{\rho ·|X_0|}$$

(15)

The desired fan $\left({\mathcal{X}}^{\prime }\stackrel{}{\leftarrow }{\mathcal{Y}}^{\prime }\stackrel{}{\to }{V}^{\mathbf{H}}\right)$ mentioned in the beginning of the section is obtained from the top fan in the diagram in (13) by conditioning on $\overline{u}\in U^N$. We will show later that for an appropriate choice of N and for most choices of $\overline{u}$, the fan we obtain in this way has the required properties.

First, we would like to make the following observations. Fix an arbitrary $\overline{u}\in U^N$. Then:

(1)

The underlying set of the probability space $B_0|\overline{u}=X_0|\overline{u}$ is $\underline{X}{}_0$.

(2)

The diagrams

$$\begin{array}{cc}\hfill {\mathcal{Y}}_{\overline{u}}^{\prime }& :=\mathcal{A}|\overline{u}\hfill \\ \hfill {\mathcal{X}}_{\overline{u}}^{\prime }& :=\mathcal{B}|\overline{u}\hfill \end{array}$$

are included in a two-fan of $\mathbf{H}$-diagrams

which is obtained by conditioning the top fan in the diagram in (13).

The very important observation is that diagrams ${\mathcal{X}}_{\overline{u}}^{\prime }|n$ and $\mathcal{X}|u$ are isomorphic for any choice of $n\in V_N$ and $u\in U$. The isomorphism is the composition of the following sequence of isomorphisms

$${\mathcal{X}}_{\overline{u}}^{\prime }|n\stackrel{}{\to }\mathcal{B}|(\overline{u},n)\stackrel{}{\to }\mathcal{A}|(\overline{u},n)\stackrel{}{\to }\mathcal{X}\left|u_n\stackrel{}{\to }\mathcal{X}\right|u$$

where the first isomorphism follows from the definition of ${\mathcal{X}}_{\overline{u}}^{\prime }$, the second—from minimality of the fan $\mathcal{B}\stackrel{}{\leftarrow }\mathcal{A}\stackrel{}{\to }{V}_N$, the third—from the definition of $\mathcal{A}$ and the fourth—from the homogeneity of $\mathcal{Z}$.

5.2. The Estimates

We now claim and prove that one could choose a number N and $\overline{u}$ in $U^N$ such that

(1)

$\mathbf{k}({\mathcal{X}}_{\overline{u}}^{\prime },\mathcal{X})\le 20\left[\left[\mathbf{H}\right]\right]$.

(2)

$\left[Y_{\overline{u},0}^{\prime }\right|X_{\overline{u},0}^{\prime }]\le 4lnln\left|X_0\right|$, where $Y_{\overline{u},0}^{\prime }$ and $X_{\overline{u},0}^{\prime }$ are initial spaces in ${\mathcal{X}}_{\overline{u}}^{\prime }$ and ${\mathcal{Y}}_{\overline{u}}^{\prime }$, respectively.

5.2.1. Total Variation and Entropic Distance Estimates

If we fix some $x_0\in X_0$, then $\nu =\nu (x_0,·)$ is a scaled binomially distributed random variable with parameters N and $\rho$, which means that $N·\nu \sim Bin(N,\rho )$.

First, we state the following bounds on the tails of a binomial distribution.

Lemma 1. 

Let ν be a scaled binomial random variable with parameters N and ρ, then

(i) 

for any $t\in [0,1]$ holds

$$\mathbb{P}\left\{|\nu -\rho |>\rho ·t\right\}\le 2·{\mathbf{e}}^{-\frac{1}{3}·N·\rho ·t^2}$$

(ii) 

for any $t\in [0,2]$ holds

$$\mathbb{P}\left\{\frac{\nu }{\rho }ln\frac{\nu }{\rho }>t\right\}\le {\mathbf{e}}^{-\frac{1}{12}·N·\rho ·t^2}$$

The proof of Lemma 1 can be found at the end of this section.

Below we use the notation $\mathbb{P}:=p_{U^N}$ for the probability distribution on $U^N$. For a pair of complete diagrams $\mathcal{C}$, ${\mathcal{C}}^{\prime }$ with the same underlying diagram of sets and with initial spaces $C_0$, $C_0^{\prime }$, we will write $\alpha (\mathcal{C},{\mathcal{C}}^{\prime })$ for the halved total variation distance between their distributions

$$\alpha (\mathcal{C},{\mathcal{C}}^{\prime }):=\frac{1}{2}{∥p_{C_0}-p_{C_0^{\prime }}∥}_1$$

Proposition 4. 

In the settings above, for $t\in [0,1]$, the following inequality holds

$$\mathbb{P}\left\{\overline{u}\in U^N:2\alpha ({\mathcal{X}}_{\overline{u}}^{\prime },\mathcal{X})>t\right\}\le 2\left|X_0\right|·{\mathbf{e}}^{-\frac{1}{3}N·\rho ·t^2}$$

Proof. 

Recall that by definition ${\mathcal{X}}_{\overline{u}}^{\prime }=\mathcal{B}|\overline{u}$. We use Equation (15) to expand the left-hand side of the inequality as follows

$$\begin{array}{l}\mathbb{P}\left\{\overline{u}\in U^N:2\alpha \left(\mathcal{B}\right|\overline{u},\mathcal{X})>t\right\}=\mathbb{P}\left\{\overline{u}\in U^N:{\displaystyle \sum _{x\in X_0}}\left|\frac{\nu (x,\overline{u})}{\rho ·|X_0|}-\frac{1}{|X_0|}\right|>t\right\}\\ =\mathbb{P}\left\{\overline{u}\in U^N:{\displaystyle \sum _{x\in X_0}}\left|\nu (x,\overline{u})-\rho \right|>\rho ·\left|X_0\right|·t\right\}\hfill \\ \le \mathbb{P}\left\{\overline{u}\in U^N:\mathrm{there}\mathrm{exists}{x}_0\mathrm{such}\mathrm{that}\left|\nu (x_0,\overline{u})-\rho \right|>\rho ·t\right\}\hfill \\ \le {\displaystyle \sum _{x\in X_0}}\mathbb{P}\left\{\overline{u}\in U^N:\left|\nu (x,\overline{u})-\rho \right|>\rho ·t\right\}\hfill \end{array}$$

Since by homogeneity of the original diagram, all the summands are the same, we can fix some $x_0\in X_0$ and estimate further:

$$\begin{array}{c}\mathbb{P}\left\{\overline{u}\in U^N:2\alpha \left(\mathcal{B}\right|\overline{u},\mathcal{X})>t\right\}\le \left|X_0\right|·\mathbb{P}\left\{\overline{u}\in U^N:\left|\nu (x_0,\overline{u})-\rho \right|>\rho ·t\right\}\hfill \end{array}$$

Applying Lemma 1(i), we obtain the required inequality. □

In the propositions below we assume that $|X_0|$ is sufficiently large (larger than ${\mathbf{e}}^{20}$).

Proposition 5. 

In the settings above and for any $\frac{10}{ln|X_0|}\le t\le 1$ holds:

$$\mathbb{P}\left\{\overline{u}\in U^N:\mathbf{k}({\mathcal{X}}_{\overline{u}}^{\prime },\mathcal{X})>t(2·\left[\left[\mathbf{G}\right]\right]·ln|X_0\left|\right)\right\}\le 2\left|X_0\right|·{\mathbf{e}}^{-\frac{1}{3}N·\rho ·t^2}$$

Proof. 

We will use local estimate, Proposition 3, to bound the entropy distance and then apply Proposition 4. To simplify notations, we will write simply $\alpha$ for $\alpha ({\mathcal{X}}_{\overline{u}}^{\prime },\mathcal{X})=\alpha \left(\mathcal{B}\right|\overline{u},\mathcal{X})$.

$$\begin{array}{c}\mathbb{P}\left\{\overline{u}\in U^N:\mathbf{k}\left(\mathcal{B}\right|\overline{u},\mathcal{X})>(2·\left[\left[\mathbf{G}\right]\right]·ln|X_0\left|\right)t\right\}\hfill \\ \le \mathbb{P}\left\{\overline{u}\in U^N:2·\left[\left[\mathbf{G}\right]\right](\alpha ·ln|X_0|+\mathsf{Ent}\left({\mathsf{\Lambda }}_{\alpha }\right))>(2·\left[\left[\mathbf{G}\right]\right]·ln|X_0\left|\right)t\right\}\hfill \\ \le \mathbb{P}\left\{\overline{u}\in U^N:\alpha +\mathsf{Ent}\left({\mathsf{\Lambda }}_{\alpha }\right)/ln\left|X_0\right|>t\right\}\hfill \end{array}$$

Please note that in the chosen regime, $t\ge 10/ln|X_0|$, the first summand in the right-hand side of the inequality is larger than the second, i.e., $\alpha \ge \mathsf{Ent}\left({\mathsf{\Lambda }}_{\alpha }\right)/ln\left|X_0\right|$ and therefore we can write

$$\begin{array}{c}\mathbb{P}\left\{\overline{u}\in U^N:\mathbf{k}\left(\mathcal{B}\right|\overline{u},\mathcal{X})>(2·\left[\left[\mathbf{G}\right]\right]·ln|X_0\left|\right)t\right\}\hfill \\ \le \mathbb{P}\left\{\overline{u}\in U^N:2\alpha >t\right\}\hfill \\ \le 2|X_0|·{\mathbf{e}}^{-\frac{1}{3}N·\rho ·t^2}\hfill \end{array}$$

  □

5.2.2. The “Height” Estimate

Recall that for given $N\in \mathbb{N}$ and $\overline{u}\in U^N$ we have constructed a two-fan of $\mathbf{H}$-diagrams

$${\mathcal{X}}_{\overline{u}}^{\prime }\stackrel{}{\leftarrow }{\mathcal{Y}}_{\overline{u}}^{\prime }\stackrel{}{\to }{V}_N^{\mathbf{H}}$$

We will now estimate the length of the arrow $Y_{\overline{u},0}^{\prime }\stackrel{}{\to }{X}_{\overline{u},0}^{\prime }$.

Proposition 6. 

In the settings above and for $t\in [0,2]$

$$\mathbb{P}\left\{\overline{u}\in U^N:\left[Y_{\overline{u},0}^{\prime }\right|X_{\overline{u},0}^{\prime }]>ln(N·\rho )+t\right\}\le \left|X_0\right|·{\mathbf{e}}^{-\frac{1}{12}N·\rho ·t^2}$$

Proof. 

First, we observe that the fiber of the reduction $Y_{\overline{u},0}^{\prime }\stackrel{}{\to }{X}_{\overline{u},0}^{\prime }$ over a point $x\in X_{\overline{u},0}^{\prime }$ is a homogeneous probability space of cardinality equal to $N(x,\overline{u})$, therefore its entropy is $lnN(x,\overline{u})$.

$$\begin{array}{c}\mathbb{P}\left\{\overline{u}\in U^N:\left[Y_{\overline{u},0}^{\prime }\right|X_{\overline{u},0}^{\prime }]>ln(N·\rho )+t\right\}\hfill \\ \mathbb{P}\left\{\overline{u}\in U^N:{\int }_{X_{\overline{u},0}^{\prime }}\left[Y_{\overline{u},0}^{\prime }\right|x]d{p}_{X_{\overline{u},0}^{\prime }}\left(x\right)>ln(N·\rho )+t\right\}\hfill \\ =\mathbb{P}\left\{\overline{u}\in U^N:\sum _{x\in X_0}\frac{\nu (x,\overline{u})}{\rho ·|X_0|}ln\left(N·\nu (x,\overline{u})\right)>ln(N·\rho )+t\right\}\hfill \\ \le \mathbb{P}\left\{\overline{u}\in U^N:\sum _{x\in X_0}\frac{\nu (x,\overline{u})}{\rho ·|X_0|}ln\left(\frac{\nu (x,\overline{u})}{\rho }\right)>t\right\}\hfill \\ \le |X_0|·\mathbb{P}\left\{\overline{u}\in U^N:\frac{\nu (x_0,\overline{u})}{\rho }ln\left(\frac{\nu (x_0,\overline{u})}{\rho }\right)>t\right\}\hfill \\ \le |X_0|·{\mathbf{e}}^{-\frac{1}{12}N·\rho ·t^2}\hfill \end{array}$$

The last inequality above follows from Lemma 1 (ii). □

5.3. Proof of Proposition 2

Let ${\mathcal{X}}_{\overline{u}}^{\prime }\stackrel{}{\leftarrow }{\mathcal{Y}}_{\overline{u}}^{\prime }\stackrel{}{\to }{V}_N$ be the fan constructed in Section 5.1. The construction is parameterized by number N and atom $\overline{u}\in U^N$. Below, we will choose a particular value for N and apply estimates in Propositions 5 and 6 with particular choice of parameter t to show that there is $\overline{u}\in U^N$, so that the fan satisfies the conclusions of Proposition 2.

Let

$$\begin{array}{l}N:={ln}^3|X_0|·{\rho }^{-1}={ln}^3\left|X_0\right|·{\mathbf{e}}^{[X_0:U]}\hfill \\ t:=\frac{10}{ln|X_0|}\hfill \end{array}$$

With these choices of N and t, Proposition 5 implies

$$\mathbb{P}\left\{\overline{u}\in U^N:\mathbf{k}({\mathcal{X}}_{\overline{u}}^{\prime },\mathcal{X})>20\left[\left[\mathbf{G}\right]\right]\right\}\le \frac{1}{4}$$

while Proposition 6 gives

$$\mathbb{P}\left\{\overline{u}\in U^N:\left[Y_{\overline{u},0}^{\prime }\right|X_{\overline{u},0}^{\prime }]>4lnln\left|X_0\right|\right\}\le \frac{1}{4}$$

Therefore, there is a choice of $\overline{u}$ such that the fan

$$\left({\mathcal{X}}^{\prime }\stackrel{}{\leftarrow }{\mathcal{Y}}^{\prime }\stackrel{}{\to }V\right):=\left({\mathcal{X}}_{\overline{u},0}^{\prime }\stackrel{}{\leftarrow }{\mathcal{Y}}_{\overline{u},0}^{\prime }\stackrel{}{\to }{V}_N\right)$$

satisfies conditions in (12). As we have explained at the beginning of Section 5.1, by collapsing isomorphism arrows, we obtain $\mathbf{G}$-diagram ${\mathcal{Z}}^{\prime }$ satisfying conclusions of Proposition 2.

5.4. Proof of Lemma 1

The Chernoff bound for the tail of a binomially distributed random variable $X\sim Bin(N,\rho )$ asserts that for any $0\le \delta \le 1$ holds

$$\begin{array}{c}\mathbb{P}\left\{X<(1-\delta )N·\rho \right\}\le {\mathbf{e}}^{-\frac{1}{2}{\delta }^{2}N·\rho }\hfill \\ \mathbb{P}\left\{X>(1+\delta )N·\rho \right\}\le {\mathbf{e}}^{-\frac{1}{3}{\delta }^{2}N·\rho }\hfill \end{array}$$

Applying the bound for the upper and lower tail for the binomially distributed random variable $N·\nu$, we obtain the inequality in (i).

The second assertion follows from the following estimate

$$\begin{array}{cc}\hfill \mathbb{P}\left\{\frac{\nu }{\rho }ln\frac{\nu }{\rho }>t\right\}& \le \mathbb{P}\left\{\frac{\nu }{\rho }\left(\frac{\nu }{\rho }-1\right)>t\right\}\hfill \\ \hfill & =\mathbb{P}\left\{\nu >\rho ·\left(\frac{\sqrt{1+4t}-1}{2}+1\right)\right\}\hfill \end{array}$$

For $0\le t\le 2$ we have $\sqrt{1+4t}-1\ge t$, therefore

$$\mathbb{P}\left\{\frac{\nu }{\rho }ln\frac{\nu }{\rho }>t\right\}\le \mathbb{P}\left\{\nu >\rho ·\left(\frac{t}{2}+1\right)\right\}$$

By the Chernoff bound, we have

$$\mathbb{P}\left\{\frac{\nu }{\rho }ln\frac{\nu }{\rho }>t\right\}\le {\mathbf{e}}^{-\frac{1}{12}N·\rho ·t^2}$$