Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.9
2.1
Journals
Entropy
Special Issues
Shannon Entropy: Mathematical View
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Shannon Entropy: Mathematical View

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (31 August 2024) | Viewed by 7942

Special Issue Editors

Prof. Dr. Josip Pečarić

E-Mail
Guest Editor
Croatian Academy of Sciences and Arts, 10000 Zagreb, Croatia
Interests: mathematical inequalities; theory of convexity; information theory; operator theory; majorization theory; statistics; functional analysis
Dr. Muhammad Adil Khan

E-Mail Website
Guest Editor
Department of Mathematics, University of Peshawar, Peshawar 25120, Pakistan
Interests: mathematical inequalities; theory of convexity; information theory; operator theory; majorization theory; functional analysis
Dr. Đilda Pečarić

E-Mail Website
Guest Editor
Department of Media and Communication, University North, 48000 Koprivnica, Croatia
Interests: scholarly communication; bibliometrics; information theory; documentation; research methods

Special Issue Information

Dear Colleagues,

The field of mathematical inequalities and their applications has recorded exponential and significant growth in the last few decades, with a considerable impact in various areas of science, such as engineering, qualitative theory of differential and integral equations, economics, computer science, mathematical statistics, information theory etc. It is noteworthy that many innovative ideas about mathematical inequalities and their applications in various areas of science can be developed by convexity. The notion of classical convexity has been streamlined by mathematical inequalities.

Information theory is the science of information, which scientifically deals with the storage, quantification and communication of information. It emerged from Claude Shannon by considering stochastic processes as a source of information. Some of the foremost quantities of information theory are the entropy, relative entropy, Zipf–Mandelbrot entropy, mutual information, Csiszar divergence, etc., which are defined as functionals of probability distributions. In turn, they characterize the behavior of long sequences of random variables and allow us to estimate the probabilities of rare events. Information theory has extensive role in communication systems with deep connections to diverse topics such as artificial intelligence, statistical mechanics, biological evolution, statistical physics, etc.

The theory of convexity and several mathematical inequalities (majorization, Jensen’s and Levinson’s inequalities, etc.) are utilized to obtain some important inequalities for various entropies (e.g., Shannon and Zipf–Mandelbrot entropies) and for divergences (e.g., Csiszar and Kullback–Leibler divergences) in information theory. As some fundamental tools, these inequalities can be used to provide error bounds while estimating the aforesaid entropies and divergences.

This Special Issue aims to provide a forum for the presentation of new and improved techniques for obtaining inequalities and their applications in information theory. In particular, this Issue will consider original and high-quality papers pertaining to inequalities for Shannon entropy, Zipf–Mandelbrot entropy, Csiszar, Jeffrey’s and Kullback–Leibler divergences, and other related topics.

Prof. Dr. Josip Pečarić
Dr. Muhammad Adil Khan
Dr. Đilda Pečarić
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • inequalities
  • theory of convexity
  • information theory
  • Shannon and Zipf–Mandelbrot entropies
  • divergences
  • applications

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (5 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

Jump to: Review

14 pages, 526 KiB  
Article
Entropy-Based Volatility Analysis of Financial Log-Returns Using Gaussian Mixture Models
by Luca Scrucca
Entropy 2024, 26(11), 907; https://doi.org/10.3390/e26110907 - 25 Oct 2024
Viewed by 571
Abstract
Volatility in financial markets refers to the variation in asset prices over time. High volatility indicates increased risk, making its evaluation essential for effective risk management. Various methods are used to assess volatility, with the standard deviation of log-returns being a common approach. [...] Read more.
Volatility in financial markets refers to the variation in asset prices over time. High volatility indicates increased risk, making its evaluation essential for effective risk management. Various methods are used to assess volatility, with the standard deviation of log-returns being a common approach. However, this implicitly assumes that log-returns follow a Gaussian distribution, which is not always valid. In this paper, we explore the use of (differential) entropy to evaluate the volatility of financial log-returns. Estimation of entropy is obtained using a Gaussian mixture model to approximate the underlying density of log-returns. Following this modeling approach, popular risk measures such as Value at Risk and Expected Shortfall can also be computed. By integrating Gaussian mixture modeling and entropy into the analysis of log-returns, we aim to provide a more accurate and robust framework for assessing financial volatility and risk measures. Full article
(This article belongs to the Special Issue Shannon Entropy: Mathematical View)
►▼ Show Figures

Figure 1

16 pages, 1608 KiB  
Article
Quantum Knowledge in Phase Space
by Davi Geiger
Entropy 2023, 25(8), 1227; https://doi.org/10.3390/e25081227 - 17 Aug 2023
Cited by 1 | Viewed by 1496
Abstract
Quantum physics through the lens of Bayesian statistics considers probability to be a degree of belief and subjective. A Bayesian derivation of the probability density function in phase space is presented. Then, a Kullback–Liebler divergence in phase space is introduced to define interference [...] Read more.
Quantum physics through the lens of Bayesian statistics considers probability to be a degree of belief and subjective. A Bayesian derivation of the probability density function in phase space is presented. Then, a Kullback–Liebler divergence in phase space is introduced to define interference and entanglement. Comparisons between each of these two quantities and the entropy are made. A brief presentation of entanglement in phase space to the spin degree of freedom and an extension to mixed states completes the work. Full article
(This article belongs to the Special Issue Shannon Entropy: Mathematical View)
►▼ Show Figures

Figure 1

29 pages, 443 KiB  
Article
Capacity-Achieving Input Distributions of Additive Vector Gaussian Noise Channels: Even-Moment Constraints and Unbounded or Compact Support
by Jonah Eisen, Ravi R. Mazumdar and Patrick Mitran
Entropy 2023, 25(8), 1180; https://doi.org/10.3390/e25081180 - 8 Aug 2023
Cited by 3 | Viewed by 1311
Abstract
We investigate the support of a capacity-achieving input to a vector-valued Gaussian noise channel. The input is subjected to a radial even-moment constraint and is either allowed to take any value in Rn or is restricted to a given compact subset of [...] Read more.
We investigate the support of a capacity-achieving input to a vector-valued Gaussian noise channel. The input is subjected to a radial even-moment constraint and is either allowed to take any value in Rn or is restricted to a given compact subset of Rn. It is shown that the support of the capacity-achieving distribution is composed of a countable union of submanifolds, each with a dimension of n−1 or less. When the input is restricted to a compact subset of Rn, this union is finite. Finally, the support of the capacity-achieving distribution is shown to have Lebesgue measure 0 and to be nowhere dense in Rn. Full article
(This article belongs to the Special Issue Shannon Entropy: Mathematical View)
27 pages, 381 KiB  
Article
Uniform Treatment of Integral Majorization Inequalities with Applications to Hermite-Hadamard-Fejér-Type Inequalities and f-Divergences
by László Horváth
Entropy 2023, 25(6), 954; https://doi.org/10.3390/e25060954 - 19 Jun 2023
Cited by 4 | Viewed by 2048
Abstract
In this paper, we present a general framework that provides a comprehensive and uniform treatment of integral majorization inequalities for convex functions and finite signed measures. Along with new results, we present unified and simple proofs of classical statements. To apply our results, [...] Read more.
In this paper, we present a general framework that provides a comprehensive and uniform treatment of integral majorization inequalities for convex functions and finite signed measures. Along with new results, we present unified and simple proofs of classical statements. To apply our results, we deal with Hermite-Hadamard-Fejér-type inequalities and their refinements. We present a general method to refine both sides of Hermite-Hadamard-Fejér-type inequalities. The results of many papers on the refinement of the Hermite-Hadamard inequality, whose proofs are based on different ideas, can be treated in a uniform way by this method. Finally, we establish a necessary and sufficient condition for when a fundamental inequality of f-divergences can be refined by another f-divergence. Full article
(This article belongs to the Special Issue Shannon Entropy: Mathematical View)

Review

Jump to: Research

29 pages, 505 KiB  
Review
An Information Theoretic Condition for Perfect Reconstruction
by Idris Delsol , Olivier Rioul , Julien Béguinot, Victor Rabiet  and Antoine Souloumiac 
Entropy 2024, 26(1), 86; https://doi.org/10.3390/e26010086 - 19 Jan 2024
Cited by 2 | Viewed by 1255
Abstract
A new information theoretic condition is presented for reconstructing a discrete random variable X based on the knowledge of a set of discrete functions of X. The reconstruction condition is derived from Shannon’s 1953 lattice theory with two entropic metrics of Shannon [...] Read more.
A new information theoretic condition is presented for reconstructing a discrete random variable X based on the knowledge of a set of discrete functions of X. The reconstruction condition is derived from Shannon’s 1953 lattice theory with two entropic metrics of Shannon and Rajski. Because such a theoretical material is relatively unknown and appears quite dispersed in different references, we first provide a synthetic description (with complete proofs) of its concepts, such as total, common, and complementary information. The definitions and properties of the two entropic metrics are also fully detailed and shown to be compatible with the lattice structure. A new geometric interpretation of such a lattice structure is then investigated, which leads to a necessary (and sometimes sufficient) condition for reconstructing the discrete random variable X given a set {X1,…,Xn} of elements in the lattice generated by X. Intuitively, the components X1,…,Xn of the original source of information X should not be globally “too far away” from X in the entropic distance in order that X is reconstructable. In other words, these components should not overall have too low of a dependence on X; otherwise, reconstruction is impossible. These geometric considerations constitute a starting point for a possible novel “perfect reconstruction theory”, which needs to be further investigated and improved along these lines. Finally, this condition is illustrated in five specific examples of perfect reconstruction problems: the reconstruction of a symmetric random variable from the knowledge of its sign and absolute value, the reconstruction of a word from a set of linear combinations, the reconstruction of an integer from its prime signature (fundamental theorem of arithmetic) and from its remainders modulo a set of coprime integers (Chinese remainder theorem), and the reconstruction of the sorting permutation of a list from a minimal set of pairwise comparisons. Full article
(This article belongs to the Special Issue Shannon Entropy: Mathematical View)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-5
Back to TopTop