Entropy

Research

21 pages, 527 KiB

Open AccessFeature PaperArticle

Bipartite Unique Neighbour Expanders via Ramanujan Graphs

by Ron Asherov and Irit Dinur

Entropy 2024, 26(4), 348; https://doi.org/10.3390/e26040348 - 20 Apr 2024

Cited by 5 | Viewed by 1479

We construct an infinite family of bounded-degree bipartite unique neighbour expander graphs with arbitrarily unbalanced sides. Although weaker than the lossless expanders constructed by Capalbo et al., our construction is simpler and may be closer to being implementable in practice, due to the [...] Read more.

We construct an infinite family of bounded-degree bipartite unique neighbour expander graphs with arbitrarily unbalanced sides. Although weaker than the lossless expanders constructed by Capalbo et al., our construction is simpler and may be closer to being implementable in practice, due to the smaller constants. We construct these graphs by composing bipartite Ramanujan graphs with a fixed-size gadget in a way that generalises the construction of unique neighbour expanders by Alon and Capalbo. For the analysis of our construction, we prove a strong upper bound on average degrees in small induced subgraphs of bipartite Ramanujan graphs. Our bound generalises Kahale’s average degree bound to bipartite Ramanujan graphs, and may be of independent interest. Surprisingly, our bound strongly relies on the exact Ramanujan-ness of the graph and is not known to hold for nearly-Ramanujan graphs. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

► Show Figures

Figure 1

18 pages, 928 KiB

Open AccessFeature PaperArticle

Side Information Design in Zero-Error Coding for Computing

by Nicolas Charpenay, Maël Le Treust and Aline Roumy

Entropy 2024, 26(4), 338; https://doi.org/10.3390/e26040338 - 16 Apr 2024

Viewed by 1438

Abstract

We investigate the zero-error coding for computing problems with encoder side information. An encoder provides access to a source X and is furnished with side information

g (Y)

. It communicates with a decoder that possesses side information Y and aims [...] Read more.

We investigate the zero-error coding for computing problems with encoder side information. An encoder provides access to a source X and is furnished with side information

g (Y)

. It communicates with a decoder that possesses side information Y and aims to retrieve

f (X, Y)

with zero probability of error, where f and g are assumed to be deterministic functions. In previous work, we determined a condition that yields an analytic expression for the optimal rate

R^{*} (g)

; in particular, it covers the case where

P_{X, Y}

is full support. In this article, we review this result and study the side information design problem, which consists of finding the best trade-offs between the quality of the encoder’s side information

g (Y)

and

R^{*} (g)

. We construct two greedy algorithms that give an achievable set of points in the side information design problem, based on partition refining and coarsening. One of them runs in polynomial time. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

► Show Figures

Figure 1

16 pages, 285 KiB

Open AccessArticle

Entropic Bounds on the Average Length of Codes with a Space

by Roberto Bruno and Ugo Vaccaro

Entropy 2024, 26(4), 283; https://doi.org/10.3390/e26040283 - 26 Mar 2024

Viewed by 994

Abstract

We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that their average length differs [...] Read more.

We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that their average length differs from the minimum possible by at most one. We obtain our results by uncovering a relation between our class of codes and the class of one-to-one codes. Additionally, we derive upper and lower bounds to the average length of optimal prefix-free codes with a space in terms of the source entropy. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

32 pages, 660 KiB

Open AccessFeature PaperArticle

Smoothing of Binary Codes, Uniform Distributions, and Applications

by Madhura Pathegama and Alexander Barg

Entropy 2023, 25(11), 1515; https://doi.org/10.3390/e25111515 - 5 Nov 2023

Cited by 2 | Viewed by 1413

Abstract

The action of a noise operator on a code transforms it into a distribution on the respective space. Some common examples from information theory include Bernoulli noise acting on a code in the Hamming space and Gaussian noise acting on a lattice in [...] Read more.

The action of a noise operator on a code transforms it into a distribution on the respective space. Some common examples from information theory include Bernoulli noise acting on a code in the Hamming space and Gaussian noise acting on a lattice in the Euclidean space. We aim to characterize the cases when the output distribution is close to the uniform distribution on the space, as measured by the Rényi divergence of order

α \in (1, \infty]

. A version of this question is known as the channel resolvability problem in information theory, and it has implications for security guarantees in wiretap channels, error correction, discrepancy, worst-to-average case complexity reductions, and many other problems. Our work quantifies the requirements for asymptotic uniformity (perfect smoothing) and identifies explicit code families that achieve it under the action of the Bernoulli and ball noise operators on the code. We derive expressions for the minimum rate of codes required to attain asymptotically perfect smoothing. In proving our results, we leverage recent results from harmonic analysis of functions on the Hamming space. Another result pertains to the use of code families in Wyner’s transmission scheme on the binary wiretap channel. We identify explicit families that guarantee strong secrecy when applied in this scheme, showing that nested Reed–Muller codes can transmit messages reliably and securely over a binary symmetric wiretap channel with a positive rate. Finally, we establish a connection between smoothing and error correction in the binary symmetric channel. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

► Show Figures

Figure 1

15 pages, 343 KiB

Open AccessFeature PaperArticle

Assisted Identification over Modulo-Additive Noise Channels

by Amos Lapidoth and Baohua Ni

Entropy 2023, 25(9), 1314; https://doi.org/10.3390/e25091314 - 8 Sep 2023

Viewed by 1089

Abstract

The gain in the identification capacity afforded by a rate-limited description of the noise sequence corrupting a modulo-additive noise channel is studied. Both the classical Ahlswede–Dueck version and the Ahlswede–Cai–Ning–Zhang version, which does not allow for missed identifications, are studied. Irrespective of whether [...] Read more.

The gain in the identification capacity afforded by a rate-limited description of the noise sequence corrupting a modulo-additive noise channel is studied. Both the classical Ahlswede–Dueck version and the Ahlswede–Cai–Ning–Zhang version, which does not allow for missed identifications, are studied. Irrespective of whether the description is provided to the receiver, to the transmitter, or to both, the two capacities coincide and both equal the helper-assisted Shannon capacity. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

10 pages, 288 KiB

Open AccessArticle

Dimension-Free Bounds for the Union-Closed Sets Conjecture

by Lei Yu

Entropy 2023, 25(5), 767; https://doi.org/10.3390/e25050767 - 8 May 2023

Cited by 6 | Viewed by 1655

Abstract

The union-closed sets conjecture states that, in any nonempty union-closed family

F

of subsets of a finite set, there exists an element contained in at least a proportion

1 / 2

of the sets of

F

. Using an information-theoretic method, Gilmer recently [...] Read more.

The union-closed sets conjecture states that, in any nonempty union-closed family

F

of subsets of a finite set, there exists an element contained in at least a proportion

1 / 2

of the sets of

F

. Using an information-theoretic method, Gilmer recently showed that there exists an element contained in at least a proportion

0.01

of the sets of such

F

. He conjectured that their technique can be pushed to the constant

\frac{3 - \sqrt{5}}{2}

which was subsequently confirmed by several researchers including Sawin. Furthermore, Sawin also showed that Gilmer’s technique can be improved to obtain a bound better than

\frac{3 - \sqrt{5}}{2}

but this new bound was not explicitly given by Sawin. This paper further improves Gilmer’s technique to derive new bounds in the optimization form for the union-closed sets conjecture. These bounds include Sawin’s improvement as a special case. By providing cardinality bounds on auxiliary random variables, we make Sawin’s improvement computable and then evaluate it numerically, which yields a bound approximately

0.38234

, slightly better than

\frac{3 - \sqrt{5}}{2} \approx 0.38197

. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

40 pages, 613 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Observations on the Lovász θ-Function, Graph Capacity, Eigenvalues, and Strong Products †

by Igal Sason

Entropy 2023, 25(1), 104; https://doi.org/10.3390/e25010104 - 4 Jan 2023

Cited by 3 | Viewed by 2660

Abstract

This paper provides new observations on the Lovász

θ

-function of graphs. These include a simple closed-form expression of that function for all strongly regular graphs, together with upper and lower bounds on that function for all regular graphs. These bounds are expressed [...] Read more.

This paper provides new observations on the Lovász

θ

-function of graphs. These include a simple closed-form expression of that function for all strongly regular graphs, together with upper and lower bounds on that function for all regular graphs. These bounds are expressed in terms of the second-largest and smallest eigenvalues of the adjacency matrix of the regular graph, together with sufficient conditions for equalities (the upper bound is due to Lovász, followed by a new sufficient condition for its tightness). These results are shown to be useful in many ways, leading to the determination of the exact value of the Shannon capacity of various graphs, eigenvalue inequalities, and bounds on the clique and chromatic numbers of graphs. Since the Lovász

θ

-function factorizes for the strong product of graphs, the results are also particularly useful for parameters of strong products or strong powers of graphs. Bounds on the smallest and second-largest eigenvalues of strong products of regular graphs are consequently derived, expressed as functions of the Lovász

θ

-function (or the smallest eigenvalue) of each factor. The resulting lower bound on the second-largest eigenvalue of a k-fold strong power of a regular graph is compared to the Alon–Boppana bound; under a certain condition, the new bound is superior in its exponential growth rate (in k). Lower bounds on the chromatic number of strong products of graphs are expressed in terms of the order and the Lovász

θ

-function of each factor. The utility of these bounds is exemplified, leading in some cases to an exact determination of the chromatic numbers of strong products or strong powers of graphs. The present research paper is aimed to have tutorial value as well. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

27 pages, 460 KiB

Open AccessFeature PaperArticle

Hypercontractive Inequalities for the Second Norm of Highly Concentrated Functions, and Mrs. Gerber’s-Type Inequalities for the Second Rényi Entropy

by Niv Levhari and Alex Samorodnitsky

Entropy 2022, 24(10), 1376; https://doi.org/10.3390/e24101376 - 27 Sep 2022

Cited by 1 | Viewed by 1463

Abstract

Let

T_{ϵ}

,

0 \leq ϵ \leq 1 / 2

, be the noise operator acting on functions on the boolean cube

{0, 1}^{n}

. Let f be a distribution on

{0, 1}^{n}

and [...] Read more.

Let

T_{ϵ}

,

0 \leq ϵ \leq 1 / 2

, be the noise operator acting on functions on the boolean cube

{0, 1}^{n}

. Let f be a distribution on

{0, 1}^{n}

and let

q > 1

. We prove tight Mrs. Gerber-type results for the second Rényi entropy of

T_{ϵ} f

which take into account the value of the

q^{t h}

Rényi entropy of f. For a general function f on

{0, 1}^{n}

we prove tight hypercontractive inequalities for the

ℓ_{2}

norm of

T_{ϵ} f

which take into account the ratio between

ℓ_{q}

and

ℓ_{1}

norms of f. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

Journal Menu

Journal Browser

Extremal and Additive Combinatorial Aspects in Information Theory

Share This Special Issue

Special Issue Editor

Special Issue Information

Benefits of Publishing in a Special Issue

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI