Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.9
2.1
Journals
Entropy
Special Issues
Information Theory for Distributed Systems
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Information Theory for Distributed Systems

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (25 July 2024) | Viewed by 12897

Special Issue Editor

Prof. Dr. Songze Li

E-Mail Website
Guest Editor
School of Cyber Science and Engineering, Southeast University, Nanjing 210018, China
Interests: coded distributed computation; privacy-preserving and trustworthy computing and learning; blockchain security and scalability
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Modern information systems, which are often distributed in nature in order to store and process massive amounts of data and information, are facing tremendous challenges related to storage/computation/communication efficiency, resource heterogeneity, privacy and security, and scalability. While information and coding theoretic techniques have conventionally been developed to improve the efficiency and reliability of communication and storage systems, they have started to find applications in addressing the challenges of modern distributed systems, including edge computing networks, content delivery networks, cloud computing systems, distributed/federated learning systems, and blockchain systems.

The aim of this Special Issue is to attract research investigations, from an information-theoretic perspective, on the fundamental limits of critical metrics in distributed systems such as computation/communication efficiency, privacy and security, accuracy, and scalability, as well as the development of information and coding theoretic tools to achieve optimal performance. Prospective authors are invited to submit original research contributions on topics including, but not limited to:

  • Coding for distributed storage systems;
  • Coded caching networks;
  • Coded distributed computation;
  • Distributed and federated learning systems;
  • Distributed private information and function retrieval;
  • Distributed ledgers/blockchains;
  • Secure multi-party computation and verifiable computation.

Prof. Dr. Songze Li
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • coded caching
  • communication bottlenecks
  • straggler/dropout resilience
  • coded distributed computation
  • privacy and security
  • distributed and federated learning
  • private information and function retrieval
  • secret sharing and threshold multi-party computation
  • verifiable computation
  • security–scalability tradeoff for blockchains

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (7 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

15 pages, 725 KiB  
Article
A Communication-Efficient Distributed Matrix Multiplication Scheme with Privacy, Security, and Resiliency
by Tao Wang, Zhiping Shi, Juan Yang and Sha Liu
Entropy 2024, 26(9), 743; https://doi.org/10.3390/e26090743 - 30 Aug 2024
Viewed by 703
Abstract
Secure distributed matrix multiplication (SDMM) schemes are crucial for distributed learning algorithms where extensive data computation is distributed across multiple servers. Inspired by the application of repairing Reed–Solomon (RS) codes in distributed storage and secret sharing, we propose SDMM schemes with reduced communication [...] Read more.
Secure distributed matrix multiplication (SDMM) schemes are crucial for distributed learning algorithms where extensive data computation is distributed across multiple servers. Inspired by the application of repairing Reed–Solomon (RS) codes in distributed storage and secret sharing, we propose SDMM schemes with reduced communication overhead through the use of trace polynomials. Specifically, these schemes are designed to address three critical concerns: (i) ensuring information-theoretic privacy against collusion among servers; (ii) providing security against Byzantine servers; and (iii) offering resiliency against stragglers to mitigate computing delays. To the best of our knowledge, security and resiliency are being considered for the first time within trace polynomial-based approaches. Furthermore, our schemes offer the advantage of reduced sub-packetization and a lower server-count requirement, which diminish the computational complexity and download cost for the user. Full article
(This article belongs to the Special Issue Information Theory for Distributed Systems)
►▼ Show Figures

Figure 1

41 pages, 536 KiB  
Article
Mechanisms for Robust Local Differential Privacy
by Milan Lopuhaä-Zwakenberg and Jasper Goseling
Entropy 2024, 26(3), 233; https://doi.org/10.3390/e26030233 - 6 Mar 2024
Cited by 1 | Viewed by 1723
Abstract
We consider privacy mechanisms for releasing data X=(S,U), where S is sensitive and U is non-sensitive. We introduce the robust local differential privacy (RLDP) framework, which provides strong privacy guarantees, while preserving utility. This is achieved [...] Read more.
We consider privacy mechanisms for releasing data X=(S,U), where S is sensitive and U is non-sensitive. We introduce the robust local differential privacy (RLDP) framework, which provides strong privacy guarantees, while preserving utility. This is achieved by providing robust privacy: our mechanisms do not only provide privacy with respect to a publicly available estimate of the unknown true distribution, but also with respect to similar distributions. Such robustness mitigates the potential privacy leaks that might arise from the difference between the true distribution and the estimated one. At the same time, we mitigate the utility penalties that come with ordinary differential privacy, which involves making worst-case assumptions and dealing with extreme cases. We achieve robustness in privacy by constructing an uncertainty set based on a Rényi divergence. By analyzing the structure of this set and approximating it with a polytope, we can use robust optimization to find mechanisms with high utility. However, this relies on vertex enumeration and becomes computationally inaccessible for large input spaces. Therefore, we also introduce two low-complexity algorithms that build on existing LDP mechanisms. We evaluate the utility and robustness of the mechanisms using numerical experiments and demonstrate that our mechanisms provide robust privacy, while achieving a utility that is close to optimal. Full article
(This article belongs to the Special Issue Information Theory for Distributed Systems)
►▼ Show Figures

Figure 1

24 pages, 1966 KiB  
Article
On the Optimal Error Exponent of Type-Based Distributed Hypothesis Testing
by Xinyi Tong, Xiangxiang Xu and Shao-Lun Huang
Entropy 2023, 25(10), 1434; https://doi.org/10.3390/e25101434 - 10 Oct 2023
Cited by 1 | Viewed by 1266
Abstract
Distributed hypothesis testing (DHT) has emerged as a significant research area, but the information-theoretic optimality of coding strategies is often typically hard to address. This paper studies the DHT problems under the type-based setting, which is requested from the popular federated learning methods. [...] Read more.
Distributed hypothesis testing (DHT) has emerged as a significant research area, but the information-theoretic optimality of coding strategies is often typically hard to address. This paper studies the DHT problems under the type-based setting, which is requested from the popular federated learning methods. Specifically, two communication models are considered: (i) DHT problem over noiseless channels, where each node observes i.i.d. samples and sends a one-dimensional statistic of observed samples to the decision center for decision making; and (ii) DHT problem over AWGN channels, where the distributed nodes are restricted to transmit functions of the empirical distributions of the observed data sequences due to practical computational constraints. For both of these problems, we present the optimal error exponent by providing both the achievability and converse results. In addition, we offer corresponding coding strategies and decision rules. Our results not only offer coding guidance for distributed systems, but also have the potential to be applied to more complex problems, enhancing the understanding and application of DHT in various domains. Full article
(This article belongs to the Special Issue Information Theory for Distributed Systems)
►▼ Show Figures

Figure 1

16 pages, 463 KiB  
Article
Multi-User PIR with Cyclic Wraparound Multi-Access Caches
by Kanishak Vaidya and Balaji Sundar Rajan
Entropy 2023, 25(8), 1228; https://doi.org/10.3390/e25081228 - 18 Aug 2023
Viewed by 1307
Abstract
We consider the problem of multi-access cache-aided multi-user Private Information Retrieval (MACAMuPIR) with cyclic wraparound cache access. In MACAMuPIR, several files are replicated across multiple servers. There are multiple users and multiple cache nodes. When the network is not congested, servers fill these [...] Read more.
We consider the problem of multi-access cache-aided multi-user Private Information Retrieval (MACAMuPIR) with cyclic wraparound cache access. In MACAMuPIR, several files are replicated across multiple servers. There are multiple users and multiple cache nodes. When the network is not congested, servers fill these cache nodes with the content of the files. During peak network traffic, each user accesses several cache nodes. Every user wants to retrieve one file from the servers but does not want the servers to know their demands. This paper proposes a private retrieval scheme for MACAMuPIR and characterizes the transmission cost for multi-access systems with cyclic wraparound cache access. We formalize privacy and correctness constraints and analyze transmission costs. The scheme outperforms the previously known dedicated cache setup, offering efficient and private retrieval. Results demonstrate the effectiveness of the multi-access approach. Our research contributes an efficient, privacy-preserving solution for multi-user PIR, advancing secure data retrieval from distributed servers. Full article
(This article belongs to the Special Issue Information Theory for Distributed Systems)
►▼ Show Figures

Figure 1

15 pages, 5872 KiB  
Article
Towards Efficient Federated Learning: Layer-Wise Pruning-Quantization Scheme and Coding Design
by Zheqi Zhu, Yuchen Shi, Gangtao Xin, Chenghui Peng, Pingyi Fan and Khaled B. Letaief
Entropy 2023, 25(8), 1205; https://doi.org/10.3390/e25081205 - 14 Aug 2023
Cited by 6 | Viewed by 1976
Abstract
As a promising distributed learning paradigm, federated learning (FL) faces the challenge of communication–computation bottlenecks in practical deployments. In this work, we mainly focus on the pruning, quantization, and coding of FL. By adopting a layer-wise operation, we propose an explicit and universal [...] Read more.
As a promising distributed learning paradigm, federated learning (FL) faces the challenge of communication–computation bottlenecks in practical deployments. In this work, we mainly focus on the pruning, quantization, and coding of FL. By adopting a layer-wise operation, we propose an explicit and universal scheme: FedLP-Q (federated learning with layer-wise pruning-quantization). Pruning strategies for homogeneity/heterogeneity scenarios, the stochastic quantization rule, and the corresponding coding scheme were developed. Both theoretical and experimental evaluations suggest that FedLP-Q improves the system efficiency of communication and computation with controllable performance degradation. The key novelty of FedLP-Q is that it serves as a joint pruning-quantization FL framework with layer-wise processing and can easily be applied in practical FL systems. Full article
(This article belongs to the Special Issue Information Theory for Distributed Systems)
►▼ Show Figures

Figure 1

16 pages, 542 KiB  
Article
Round-Efficient Secure Inference Based on Masked Secret Sharing for Quantized Neural Network
by Weiming Wei, Chunming Tang and Yucheng Chen
Entropy 2023, 25(2), 389; https://doi.org/10.3390/e25020389 - 20 Feb 2023
Cited by 1 | Viewed by 2009
Abstract
Existing secure multiparty computation protocol from secret sharing is usually under this assumption of the fast network, which limits the practicality of the scheme on the low bandwidth and high latency network. A proven method is to reduce the communication rounds of the [...] Read more.
Existing secure multiparty computation protocol from secret sharing is usually under this assumption of the fast network, which limits the practicality of the scheme on the low bandwidth and high latency network. A proven method is to reduce the communication rounds of the protocol as much as possible or construct a constant-round protocol. In this work, we provide a series of constant-round secure protocols for quantized neural network (QNN) inference. This is given by masked secret sharing (MSS) in the three-party honest-majority setting. Our experiment shows that our protocol is practical and suitable for low-bandwidth and high-latency networks. To the best of our knowledge, this work is the first one where the QNN inference based on masked secret sharing is implemented. Full article
(This article belongs to the Special Issue Information Theory for Distributed Systems)
►▼ Show Figures

Figure 1

33 pages, 752 KiB  
Article
Distributed Hypothesis Testing over a Noisy Channel: Error-Exponents Trade-Off
by Sreejith Sreekumar and Deniz Gündüz
Entropy 2023, 25(2), 304; https://doi.org/10.3390/e25020304 - 6 Feb 2023
Viewed by 2103
Abstract
A two-terminal distributed binary hypothesis testing problem over a noisy channel is studied. The two terminals, called the observer and the decision maker, each has access to n independent and identically distributed samples, denoted by U and V, respectively. The observer communicates [...] Read more.
A two-terminal distributed binary hypothesis testing problem over a noisy channel is studied. The two terminals, called the observer and the decision maker, each has access to n independent and identically distributed samples, denoted by U and V, respectively. The observer communicates to the decision maker over a discrete memoryless channel, and the decision maker performs a binary hypothesis test on the joint probability distribution of (U,V) based on V and the noisy information received from the observer. The trade-off between the exponents of the type I and type II error probabilities is investigated. Two inner bounds are obtained, one using a separation-based scheme that involves type-based compression and unequal error-protection channel coding, and the other using a joint scheme that incorporates type-based hybrid coding. The separation-based scheme is shown to recover the inner bound obtained by Han and Kobayashi for the special case of a rate-limited noiseless channel, and also the one obtained by the authors previously for a corner point of the trade-off. Finally, we show via an example that the joint scheme achieves a strictly tighter bound than the separation-based scheme for some points of the error-exponents trade-off. Full article
(This article belongs to the Special Issue Information Theory for Distributed Systems)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-7
Back to TopTop