Entropy

16 pages, 15341 KB

Open AccessEditor’s ChoiceArticle

A Geometric Perspective on Information Plane Analysis

by Mina Basirat, Bernhard C. Geiger and Peter M. Roth

Entropy 2021, 23(6), 711; https://doi.org/10.3390/e23060711 - 3 Jun 2021

Cited by 4 | Viewed by 4681

Information plane analysis, describing the mutual information between the input and a hidden layer and between a hidden layer and the target over time, has recently been proposed to analyze the training of neural networks. Since the activations of a hidden layer are [...] Read more.

Information plane analysis, describing the mutual information between the input and a hidden layer and between a hidden layer and the target over time, has recently been proposed to analyze the training of neural networks. Since the activations of a hidden layer are typically continuous-valued, this mutual information cannot be computed analytically and must thus be estimated, resulting in apparently inconsistent or even contradicting results in the literature. The goal of this paper is to demonstrate how information plane analysis can still be a valuable tool for analyzing neural network training. To this end, we complement the prevailing binning estimator for mutual information with a geometric interpretation. With this geometric interpretation in mind, we evaluate the impact of regularization and interpret phenomena such as underfitting and overfitting. In addition, we investigate neural network learning in the presence of noisy data and noisy labels. Full article

► Show Figures

Figure 1

21 pages, 3269 KB

Open AccessEditor’s ChoiceArticle

Disentangling the Information in Species Interaction Networks

by Michiel Stock, Laura Hoebeke and Bernard De Baets

Entropy 2021, 23(6), 703; https://doi.org/10.3390/e23060703 - 2 Jun 2021

Viewed by 4488

Abstract

Shannon’s entropy measure is a popular means for quantifying ecological diversity. We explore how one can use information-theoretic measures (that are often called indices in ecology) on joint ensembles to study the diversity of species interaction networks. We leverage the little-known balance equation [...] Read more.

Shannon’s entropy measure is a popular means for quantifying ecological diversity. We explore how one can use information-theoretic measures (that are often called indices in ecology) on joint ensembles to study the diversity of species interaction networks. We leverage the little-known balance equation to decompose the network information into three components describing the species abundance, specificity, and redundancy. This balance reveals that there exists a fundamental trade-off between these components. The decomposition can be straightforwardly extended to analyse networks through time as well as space, leading to the corresponding notions for alpha, beta, and gamma diversity. Our work aims to provide an accessible introduction for ecologists. To this end, we illustrate the interpretation of the components on numerous real networks. The corresponding code is made available to the community in the specialised Julia package EcologicalNetworks.jl. Full article

(This article belongs to the Special Issue Information Theory-Based Approach to Assessing Ecosystem)

► Show Figures

Figure 1

27 pages, 3083 KB

Open AccessEditor’s ChoiceArticle

Information and Self-Organization II: Steady State and Phase Transition

by Hermann Haken and Juval Portugali

Entropy 2021, 23(6), 707; https://doi.org/10.3390/e23060707 - 2 Jun 2021

Cited by 25 | Viewed by 6127

Abstract

This paper starts from Schrödinger’s famous question “what is life” and elucidates answers that invoke, in particular, Friston’s free energy principle and its relation to the method of Bayesian inference and to Synergetics 2nd foundation that utilizes Jaynes’ maximum entropy principle. Our presentation [...] Read more.

This paper starts from Schrödinger’s famous question “what is life” and elucidates answers that invoke, in particular, Friston’s free energy principle and its relation to the method of Bayesian inference and to Synergetics 2nd foundation that utilizes Jaynes’ maximum entropy principle. Our presentation reflects the shift from the emphasis on physical principles to principles of information theory and Synergetics. In view of the expected general audience of this issue, we have chosen a somewhat tutorial style that does not require special knowledge on physics but familiarizes the reader with concepts rooted in information theory and Synergetics. Full article

(This article belongs to the Special Issue Information and Self-Organization II)

► Show Figures

Figure 1

68 pages, 3761 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Quantum Foundations of Classical Reversible Computing

by Michael P. Frank and Karpur Shukla

Entropy 2021, 23(6), 701; https://doi.org/10.3390/e23060701 - 1 Jun 2021

Cited by 8 | Viewed by 10857

Abstract

The reversible computation paradigm aims to provide a new foundation for general classical digital computing that is capable of circumventing the thermodynamic limits to the energy efficiency of the conventional, non-reversible digital paradigm. However, to date, the essential rationale for, and analysis of, [...] Read more.

The reversible computation paradigm aims to provide a new foundation for general classical digital computing that is capable of circumventing the thermodynamic limits to the energy efficiency of the conventional, non-reversible digital paradigm. However, to date, the essential rationale for, and analysis of, classical reversible computing (RC) has not yet been expressed in terms that leverage the modern formal methods of non-equilibrium quantum thermodynamics (NEQT). In this paper, we begin developing an NEQT-based foundation for the physics of reversible computing. We use the framework of Gorini-Kossakowski-Sudarshan-Lindblad dynamics (a.k.a. Lindbladians) with multiple asymptotic states, incorporating recent results from resource theory, full counting statistics and stochastic thermodynamics. Important conclusions include that, as expected: (1) Landauer’s Principle indeed sets a strict lower bound on entropy generation in traditional non-reversible architectures for deterministic computing machines when we account for the loss of correlations; and (2) implementations of the alternative reversible computation paradigm can potentially avoid such losses, and thereby circumvent the Landauer limit, potentially allowing the efficiency of future digital computing technologies to continue improving indefinitely. We also outline a research plan for identifying the fundamental minimum energy dissipation of reversible computing machines as a function of speed. Full article

(This article belongs to the Special Issue Physical Information and the Physical Foundations of Computation)

► Show Figures

Figure 1

23 pages, 5917 KB

Open AccessEditor’s ChoiceArticle

Information Geometric Theory in the Prediction of Abrupt Changes in System Dynamics

by Adrian-Josue Guel-Cortez and Eun-jin Kim

Entropy 2021, 23(6), 694; https://doi.org/10.3390/e23060694 - 31 May 2021

Cited by 22 | Viewed by 4394

Abstract

Detection and measurement of abrupt changes in a process can provide us with important tools for decision making in systems management. In particular, it can be utilised to predict the onset of a sudden event such as a rare, extreme event which causes [...] Read more.

Detection and measurement of abrupt changes in a process can provide us with important tools for decision making in systems management. In particular, it can be utilised to predict the onset of a sudden event such as a rare, extreme event which causes the abrupt dynamical change in the system. Here, we investigate the prediction capability of information theory by focusing on how sensitive information-geometric theory (information length diagnostics) and entropy-based information theoretical method (information flow) are to abrupt changes. To this end, we utilise a non-autonomous Kramer equation by including a sudden perturbation to the system to mimic the onset of a sudden event and calculate time-dependent probability density functions (PDFs) and various statistical quantities with the help of numerical simulations. We show that information length diagnostics predict the onset of a sudden event better than the information flow. Furthermore, it is explicitly shown that the information flow like any other entropy-based measures has limitations in measuring perturbations which do not affect entropy. Full article

(This article belongs to the Special Issue Statistical Theory and Modeling of Rare, Extreme Events: Entropy and Information Theory)

► Show Figures

Figure 1

25 pages, 6027 KB

Open AccessEditor’s ChoiceArticle

Medium Entropy Reduction and Instability in Stochastic Systems with Distributed Delay

by Sarah A. M. Loos, Simon Hermann and Sabine H. L. Klapp

Entropy 2021, 23(6), 696; https://doi.org/10.3390/e23060696 - 31 May 2021

Cited by 11 | Viewed by 6269

Abstract

Many natural and artificial systems are subject to some sort of delay, which can be in the form of a single discrete delay or distributed over a range of times. Here, we discuss the impact of this distribution on (thermo-)dynamical properties of time-delayed [...] Read more.

Many natural and artificial systems are subject to some sort of delay, which can be in the form of a single discrete delay or distributed over a range of times. Here, we discuss the impact of this distribution on (thermo-)dynamical properties of time-delayed stochastic systems. To this end, we study a simple classical model with white and colored noise, and focus on the class of Gamma-distributed delays which includes a variety of distinct delay distributions typical for feedback experiments and biological systems. A physical application is a colloid subject to time-delayed feedback control, which is, in principle, experimentally realizable by co-moving optical traps. We uncover several unexpected phenomena in regard to the system’s linear stability and its thermodynamic properties. First, increasing the mean delay time can destabilize or stabilize the process, depending on the distribution of the delay. Second, for all considered distributions, the heat dissipated by the controlled system (e.g., the colloidal particle) can become negative, which implies that the delay force extracts energy and entropy of the bath. As we show here, this refrigerating effect is particularly pronounced for exponential delay. For a specific non-reciprocal realization of a control device, we find that the entropic costs, measured by the total entropy production of the system plus controller, are the lowest for exponential delay. The exponential delay further yields the largest stable parameter regions. In this sense, exponential delay represents the most effective and robust type of delayed feedback. Full article

(This article belongs to the Special Issue Nonequilibrium Thermodynamics and Stochastic Processes)

► Show Figures

Graphical abstract

22 pages, 2352 KB

Open AccessEditor’s ChoiceArticle

Socio-Economic Impact of the Covid-19 Pandemic in the U.S.

by Jonathan Barlow and Irena Vodenska

Entropy 2021, 23(6), 673; https://doi.org/10.3390/e23060673 - 27 May 2021

Cited by 23 | Viewed by 8132

Abstract

This paper proposes a dynamic cascade model to investigate the systemic risk posed by sector-level industries within the U.S. inter-industry network. We then use this model to study the effect of the disruptions presented by Covid-19 on the U.S. economy. We construct a [...] Read more.

This paper proposes a dynamic cascade model to investigate the systemic risk posed by sector-level industries within the U.S. inter-industry network. We then use this model to study the effect of the disruptions presented by Covid-19 on the U.S. economy. We construct a weighted digraph G = (V,E,W) using the industry-by-industry total requirements table for 2018, provided by the Bureau of Economic Analysis (BEA). We impose an initial shock that disrupts the production capacity of one or more industries, and we calculate the propagation of production shortages with a modified Cobb–Douglas production function. For the Covid-19 case, we model the initial shock based on the loss of labor between March and April 2020 as reported by the Bureau of Labor Statistics (BLS). The industries within the network are assigned a resilience that determines the ability of an industry to absorb input losses, such that if the rate of input loss exceeds the resilience, the industry fails, and its outputs go to zero. We observed a critical resilience, such that, below this critical value, the network experienced a catastrophic cascade resulting in total network collapse. Lastly, we model the economic recovery from June 2020 through March 2021 using BLS data. Full article

(This article belongs to the Special Issue Structures and Dynamics of Economic Complex Networks)

► Show Figures

Figure 1

19 pages, 2773 KB

Open AccessEditor’s ChoiceArticle

Gradient Profile Estimation Using Exponential Cubic Spline Smoothing in a Bayesian Framework

by Kushani De Silva, Carlo Cafaro and Adom Giffin

Entropy 2021, 23(6), 674; https://doi.org/10.3390/e23060674 - 27 May 2021

Viewed by 3887

Abstract

Attaining reliable gradient profiles is of utmost relevance for many physical systems. In many situations, the estimation of the gradient is inaccurate due to noise. It is common practice to first estimate the underlying system and then compute the gradient profile by taking [...] Read more.

Attaining reliable gradient profiles is of utmost relevance for many physical systems. In many situations, the estimation of the gradient is inaccurate due to noise. It is common practice to first estimate the underlying system and then compute the gradient profile by taking the subsequent analytic derivative of the estimated system. The underlying system is often estimated by fitting or smoothing the data using other techniques. Taking the subsequent analytic derivative of an estimated function can be ill-posed. This becomes worse as the noise in the system increases. As a result, the uncertainty generated in the gradient estimate increases. In this paper, a theoretical framework for a method to estimate the gradient profile of discrete noisy data is presented. The method was developed within a Bayesian framework. Comprehensive numerical experiments were conducted on synthetic data at different levels of noise. The accuracy of the proposed method was quantified. Our findings suggest that the proposed gradient profile estimation method outperforms the state-of-the-art methods. Full article

(This article belongs to the Collection Advances in Applied Statistical Mechanics)

► Show Figures

Figure 1

12 pages, 2145 KB

Open AccessEditor’s ChoiceArticle

Stochastic Thermodynamics of a Piezoelectric Energy Harvester Model

by Luigi Costanzo, Alessandro Lo Schiavo, Alessandro Sarracino and Massimo Vitelli

Entropy 2021, 23(6), 677; https://doi.org/10.3390/e23060677 - 27 May 2021

Cited by 15 | Viewed by 4121

Abstract

We experimentally study a piezoelectric energy harvester driven by broadband random vibrations. We show that a linear model, consisting of an underdamped Langevin equation for the dynamics of the tip mass, electromechanically coupled with a capacitor and a load resistor, can accurately describe [...] Read more.

We experimentally study a piezoelectric energy harvester driven by broadband random vibrations. We show that a linear model, consisting of an underdamped Langevin equation for the dynamics of the tip mass, electromechanically coupled with a capacitor and a load resistor, can accurately describe the experimental data. In particular, the theoretical model allows us to define fluctuating currents and to study the stochastic thermodynamics of the system, with focus on the distribution of the extracted work over different time intervals. Our analytical and numerical analysis of the linear model is succesfully compared to the experiments. Full article

(This article belongs to the Special Issue Free Energy and Entropy Changes: From Molecular Dynamics Simulations to the Developing Theories of Small Systems)

► Show Figures

Figure 1

30 pages, 2012 KB

Open AccessEditor’s ChoiceArticle

A Maximum Entropy Model of Bounded Rational Decision-Making with Prior Beliefs and Market Feedback

by Benjamin Patrick Evans and Mikhail Prokopenko

Entropy 2021, 23(6), 669; https://doi.org/10.3390/e23060669 - 26 May 2021

Cited by 18 | Viewed by 7694

Abstract

Bounded rationality is an important consideration stemming from the fact that agents often have limits on their processing abilities, making the assumption of perfect rationality inapplicable to many real tasks. We propose an information-theoretic approach to the inference of agent decisions under Smithian [...] Read more.

Bounded rationality is an important consideration stemming from the fact that agents often have limits on their processing abilities, making the assumption of perfect rationality inapplicable to many real tasks. We propose an information-theoretic approach to the inference of agent decisions under Smithian competition. The model explicitly captures the boundedness of agents (limited in their information-processing capacity) as the cost of information acquisition for expanding their prior beliefs. The expansion is measured as the Kullblack–Leibler divergence between posterior decisions and prior beliefs. When information acquisition is free, the homo economicus agent is recovered, while in cases when information acquisition becomes costly, agents instead revert to their prior beliefs. The maximum entropy principle is used to infer least biased decisions based upon the notion of Smithian competition formalised within the Quantal Response Statistical Equilibrium framework. The incorporation of prior beliefs into such a framework allowed us to systematically explore the effects of prior beliefs on decision-making in the presence of market feedback, as well as importantly adding a temporal interpretation to the framework. We verified the proposed model using Australian housing market data, showing how the incorporation of prior knowledge alters the resulting agent decisions. Specifically, it allowed for the separation of past beliefs and utility maximisation behaviour of the agent as well as the analysis into the evolution of agent beliefs. Full article

(This article belongs to the Special Issue Three Risky Decades: A Time for Econophysics?)

► Show Figures

Figure 1

37 pages, 1220 KB

Open AccessEditor’s ChoiceArticle

Ordinal Pattern Dependence in the Context of Long-Range Dependence

by Ines Nüßgen and Alexander Schnurr

Entropy 2021, 23(6), 670; https://doi.org/10.3390/e23060670 - 26 May 2021

Cited by 5 | Viewed by 3816

Abstract

Ordinal pattern dependence is a multivariate dependence measure based on the co-movement of two time series. In strong connection to ordinal time series analysis, the ordinal information is taken into account to derive robust results on the dependence between the two processes. This [...] Read more.

Ordinal pattern dependence is a multivariate dependence measure based on the co-movement of two time series. In strong connection to ordinal time series analysis, the ordinal information is taken into account to derive robust results on the dependence between the two processes. This article deals with ordinal pattern dependence for a long-range dependent time series including mixed cases of short- and long-range dependence. We investigate the limit distributions for estimators of ordinal pattern dependence. In doing so, we point out the differences that arise for the underlying time series having different dependence structures. Depending on these assumptions, central and non-central limit theorems are proven. The limit distributions for the latter ones can be included in the class of multivariate Rosenblatt processes. Finally, a simulation study is provided to illustrate our theoretical findings. Full article

(This article belongs to the Special Issue Time Series Modelling)

► Show Figures

Figure 1

12 pages, 630 KB

Open AccessEditor’s ChoiceArticle

Menzerath’s Law in the Syntax of Languages Compared with Random Sentences

by Kumiko Tanaka-Ishii

Entropy 2021, 23(6), 661; https://doi.org/10.3390/e23060661 - 25 May 2021

Cited by 12 | Viewed by 4069

Abstract

The Menzerath law is considered to show an aspect of the complexity underlying natural language. This law suggests that, for a linguistic unit, the size (y) of a linguistic construct decreases as the number (x) of constructs in the [...] Read more.

The Menzerath law is considered to show an aspect of the complexity underlying natural language. This law suggests that, for a linguistic unit, the size (y) of a linguistic construct decreases as the number (x) of constructs in the unit increases. This article investigates this property syntactically, with x as the number of constituents modifying the main predicate of a sentence and y as the size of those constituents in terms of the number of words. Following previous articles that demonstrated that the Menzerath property held for dependency corpora, such as in Czech and Ukrainian, this article first examines how well the property applies across languages by using the entire Universal Dependency dataset ver. 2.3, including 76 languages over 129 corpora and the Penn Treebank (PTB). The results show that the law holds reasonably well for

x > 2

. Then, for comparison, the property is investigated with syntactically randomized sentences generated from the PTB. These results show that the property is almost reproducible even from simple random data. Further analysis of the property highlights more detailed characteristics of natural language. Full article

(This article belongs to the Section Complexity)

► Show Figures

Figure 1

18 pages, 2185 KB

Open AccessEditor’s ChoiceArticle

Shall I Work with Them? A Knowledge Graph-Based Approach for Predicting Future Research Collaborations

by Nikos Kanakaris, Nikolaos Giarelis, Ilias Siachos and Nikos Karacapilidis

Entropy 2021, 23(6), 664; https://doi.org/10.3390/e23060664 - 25 May 2021

Cited by 12 | Viewed by 5770

Abstract

We consider the prediction of future research collaborations as a link prediction problem applied on a scientific knowledge graph. To the best of our knowledge, this is the first work on the prediction of future research collaborations that combines structural and textual information [...] Read more.

We consider the prediction of future research collaborations as a link prediction problem applied on a scientific knowledge graph. To the best of our knowledge, this is the first work on the prediction of future research collaborations that combines structural and textual information of a scientific knowledge graph through a purposeful integration of graph algorithms and natural language processing techniques. Our work: (i) investigates whether the integration of unstructured textual data into a single knowledge graph affects the performance of a link prediction model, (ii) studies the effect of previously proposed graph kernels based approaches on the performance of an ML model, as far as the link prediction problem is concerned, and (iii) proposes a three-phase pipeline that enables the exploitation of structural and textual information, as well as of pre-trained word embeddings. We benchmark the proposed approach against classical link prediction algorithms using accuracy, recall, and precision as our performance metrics. Finally, we empirically test our approach through various feature combinations with respect to the link prediction problem. Our experimentations with the new COVID-19 Open Research Dataset demonstrate a significant improvement of the abovementioned performance metrics in the prediction of future research collaborations. Full article

► Show Figures

Figure 1

14 pages, 2907 KB

Open AccessFeature PaperEditor’s ChoiceArticle

A Thermodynamic Approach to Measuring Entropy in a Few-Electron Nanodevice

by Eugenia Pyurbeeva and Jan A. Mol

Entropy 2021, 23(6), 640; https://doi.org/10.3390/e23060640 - 21 May 2021

Cited by 18 | Viewed by 5710

Abstract

The entropy of a system gives a powerful insight into its microscopic degrees of freedom; however, standard experimental ways of measuring entropy through heat capacity are hard to apply to nanoscale systems, as they require the measurement of increasingly small amounts of heat. [...] Read more.

The entropy of a system gives a powerful insight into its microscopic degrees of freedom; however, standard experimental ways of measuring entropy through heat capacity are hard to apply to nanoscale systems, as they require the measurement of increasingly small amounts of heat. Two alternative entropy measurement methods have been recently proposed for nanodevices: through charge balance measurements and transport properties. We describe a self-consistent thermodynamic framework for applying thermodynamic relations to few-electron nanodevices—small systems, where fluctuations in particle number are significant, whilst highlighting several ongoing misconceptions. We derive a relation (a consequence of a Maxwell relation for small systems), which describes both existing entropy measurement methods as special cases, while also allowing the experimentalist to probe the intermediate regime between them. Finally, we independently prove the applicability of our framework in systems with complex microscopic dynamics—those with many excited states of various degeneracies—from microscopic considerations. Full article

(This article belongs to the Special Issue Nature of Entropy and Its Direct Metrology)

► Show Figures

Figure 1

14 pages, 3710 KB

Open AccessEditor’s ChoiceArticle

Characterization of a Two-Photon Quantum Battery: Initial Conditions, Stability and Work Extraction

by Anna Delmonte, Alba Crescente, Matteo Carrega, Dario Ferraro and Maura Sassetti

Entropy 2021, 23(5), 612; https://doi.org/10.3390/e23050612 - 14 May 2021

Cited by 69 | Viewed by 5809

Abstract

We consider a quantum battery that is based on a two-level system coupled with a cavity radiation by means of a two-photon interaction. Various figures of merit, such as stored energy, average charging power, energy fluctuations, and extractable work are investigated, considering, as [...] Read more.

We consider a quantum battery that is based on a two-level system coupled with a cavity radiation by means of a two-photon interaction. Various figures of merit, such as stored energy, average charging power, energy fluctuations, and extractable work are investigated, considering, as possible initial conditions for the cavity, a Fock state, a coherent state, and a squeezed state. We show that the first state leads to better performances for the battery. However, a coherent state with the same average number of photons, even if it is affected by stronger fluctuations in the stored energy, results in quite interesting performance, in particular since it allows for almost completely extracting the stored energy as usable work at short enough times. Full article

(This article belongs to the Special Issue Non-equilibrium Thermodynamics in the Quantum Regime)

► Show Figures

Graphical abstract

18 pages, 2418 KB

Open AccessEditor’s ChoiceArticle

Adding Prior Information in FWI through Relative Entropy

by Danilo Santos Cruz, João M. de Araújo, Carlos A. N. da Costa and Carlos C. N. da Silva

Entropy 2021, 23(5), 599; https://doi.org/10.3390/e23050599 - 13 May 2021

Cited by 4 | Viewed by 3210

Abstract

Full waveform inversion is an advantageous technique for obtaining high-resolution subsurface information. In the petroleum industry, mainly in reservoir characterisation, it is common to use information from wells as previous information to decrease the ambiguity of the obtained results. For this, we propose [...] Read more.

Full waveform inversion is an advantageous technique for obtaining high-resolution subsurface information. In the petroleum industry, mainly in reservoir characterisation, it is common to use information from wells as previous information to decrease the ambiguity of the obtained results. For this, we propose adding a relative entropy term to the formalism of the full waveform inversion. In this context, entropy will be just a nomenclature for regularisation and will have the role of helping the converge to the global minimum. The application of entropy in inverse problems usually involves formulating the problem, so that it is possible to use statistical concepts. To avoid this step, we propose a deterministic application to the full waveform inversion. We will discuss some aspects of relative entropy and show three different ways of using them to add prior information through entropy in the inverse problem. We use a dynamic weighting scheme to add prior information through entropy. The idea is that the prior information can help to find the path of the global minimum at the beginning of the inversion process. In all cases, the prior information can be incorporated very quickly into the full waveform inversion and lead the inversion to the desired solution. When we include the logarithmic weighting that constitutes entropy to the inverse problem, we will suppress the low-intensity ripples and sharpen the point events. Thus, the addition of entropy relative to full waveform inversion can provide a result with better resolution. In regions where salt is present in the BP 2004 model, we obtained a significant improvement by adding prior information through the relative entropy for synthetic data. We will show that the prior information added through entropy in full-waveform inversion formalism will prove to be a way to avoid local minimums. Full article

(This article belongs to the Special Issue Entropy in Image Analysis III)

► Show Figures

Figure 1

33 pages, 2474 KB

Open AccessEditor’s ChoiceArticle

Information Structures for Causally Explainable Decisions

by Louis Anthony Cox, Jr.

Entropy 2021, 23(5), 601; https://doi.org/10.3390/e23050601 - 13 May 2021

Cited by 9 | Viewed by 5663

Abstract

For an AI agent to make trustworthy decision recommendations under uncertainty on behalf of human principals, it should be able to explain why its recommended decisions make preferred outcomes more likely and what risks they entail. Such rationales use causal models to link [...] Read more.

For an AI agent to make trustworthy decision recommendations under uncertainty on behalf of human principals, it should be able to explain why its recommended decisions make preferred outcomes more likely and what risks they entail. Such rationales use causal models to link potential courses of action to resulting outcome probabilities. They reflect an understanding of possible actions, preferred outcomes, the effects of action on outcome probabilities, and acceptable risks and trade-offs—the standard ingredients of normative theories of decision-making under uncertainty, such as expected utility theory. Competent AI advisory systems should also notice changes that might affect a user’s plans and goals. In response, they should apply both learned patterns for quick response (analogous to fast, intuitive “System 1” decision-making in human psychology) and also slower causal inference and simulation, decision optimization, and planning algorithms (analogous to deliberative “System 2” decision-making in human psychology) to decide how best to respond to changing conditions. Concepts of conditional independence, conditional probability tables (CPTs) or models, causality, heuristic search for optimal plans, uncertainty reduction, and value of information (VoI) provide a rich, principled framework for recognizing and responding to relevant changes and features of decision problems via both learned and calculated responses. This paper reviews how these and related concepts can be used to identify probabilistic causal dependencies among variables, detect changes that matter for achieving goals, represent them efficiently to support responses on multiple time scales, and evaluate and update causal models and plans in light of new data. The resulting causally explainable decisions make efficient use of available information to achieve goals in uncertain environments. Full article

(This article belongs to the Special Issue Information Theory at the Crossroads of Artificial Intelligence, Human Cognition, and Economics)

► Show Figures

Figure 1

18 pages, 7262 KB

Open AccessEditor’s ChoiceArticle

EEG Fractal Analysis Reflects Brain Impairment after Stroke

by Maria Rubega, Emanuela Formaggio, Franco Molteni, Eleonora Guanziroli, Roberto Di Marco, Claudio Baracchini, Mario Ermani, Nick S. Ward, Stefano Masiero and Alessandra Del Felice

Entropy 2021, 23(5), 592; https://doi.org/10.3390/e23050592 - 11 May 2021

Cited by 21 | Viewed by 5659

Abstract

Stroke is the commonest cause of disability. Novel treatments require an improved understanding of the underlying mechanisms of recovery. Fractal approaches have demonstrated that a single metric can describe the complexity of seemingly random fluctuations of physiological signals. We hypothesize that fractal algorithms [...] Read more.

Stroke is the commonest cause of disability. Novel treatments require an improved understanding of the underlying mechanisms of recovery. Fractal approaches have demonstrated that a single metric can describe the complexity of seemingly random fluctuations of physiological signals. We hypothesize that fractal algorithms applied to electroencephalographic (EEG) signals may track brain impairment after stroke. Sixteen stroke survivors were studied in the hyperacute (<48 h) and in the acute phase (∼1 week after stroke), and 35 stroke survivors during the early subacute phase (from 8 days to 32 days and after ∼2 months after stroke): We compared resting-state EEG fractal changes using fractal measures (i.e., Higuchi Index, Tortuosity) with 11 healthy controls. Both Higuchi index and Tortuosity values were significantly lower after a stroke throughout the acute and early subacute stage compared to healthy subjects, reflecting a brain activity which is significantly less complex. These indices may be promising metrics to track behavioral changes in the very early stage after stroke. Our findings might contribute to the neurorehabilitation quest in identifying reliable biomarkers for a better tailoring of rehabilitation pathways. Full article

(This article belongs to the Special Issue Fractal and Multifractal Analysis of Complex Networks)

► Show Figures

Graphical abstract

19 pages, 533 KB

Open AccessEditor’s ChoiceArticle

Uncertainty Relation between Detection Probability and Energy Fluctuations

by Felix Thiel, Itay Mualem, David Kessler and Eli Barkai

Entropy 2021, 23(5), 595; https://doi.org/10.3390/e23050595 - 11 May 2021

Cited by 7 | Viewed by 3408

Abstract

A classical random walker starting on a node of a finite graph will always reach any other node since the search is ergodic, namely it fully explores space, hence the arrival probability is unity. For quantum walks, destructive interference may induce effectively non-ergodic [...] Read more.

A classical random walker starting on a node of a finite graph will always reach any other node since the search is ergodic, namely it fully explores space, hence the arrival probability is unity. For quantum walks, destructive interference may induce effectively non-ergodic features in such search processes. Under repeated projective local measurements, made on a target state, the final detection of the system is not guaranteed since the Hilbert space is split into a bright subspace and an orthogonal dark one. Using this we find an uncertainty relation for the deviations of the detection probability from its classical counterpart, in terms of the energy fluctuations. Full article

(This article belongs to the Special Issue Axiomatic Approaches to Quantum Mechanics)

► Show Figures

Figure 1

23 pages, 1694 KB

Open AccessEditor’s ChoiceArticle

Unveiling Operator Growth Using Spin Correlation Functions

by Matteo Carrega, Joonho Kim and Dario Rosa

Entropy 2021, 23(5), 587; https://doi.org/10.3390/e23050587 - 10 May 2021

Cited by 16 | Viewed by 4695

Abstract

In this paper, we study non-equilibrium dynamics induced by a sudden quench of strongly correlated Hamiltonians with all-to-all interactions. By relying on a Sachdev-Ye-Kitaev (SYK)-based quench protocol, we show that the time evolution of simple spin-spin correlation functions is highly sensitive to the [...] Read more.

In this paper, we study non-equilibrium dynamics induced by a sudden quench of strongly correlated Hamiltonians with all-to-all interactions. By relying on a Sachdev-Ye-Kitaev (SYK)-based quench protocol, we show that the time evolution of simple spin-spin correlation functions is highly sensitive to the degree of k-locality of the corresponding operators, once an appropriate set of fundamental fields is identified. By tracking the time-evolution of specific spin-spin correlation functions and their decay, we argue that it is possible to distinguish between operator-hopping and operator growth dynamics; the latter being a hallmark of quantum chaos in many-body quantum systems. Such an observation, in turn, could constitute a promising tool to probe the emergence of chaotic behavior, rather accessible in state-of-the-art quench setups. Full article

(This article belongs to the Special Issue Entropy and Complexity in Quantum Dynamics)

► Show Figures

Figure 1

26 pages, 343 KB

Open AccessEditor’s ChoiceReview

The Phase Space Model of Nonrelativistic Quantum Mechanics

by Jaromir Tosiek and Maciej Przanowski

Entropy 2021, 23(5), 581; https://doi.org/10.3390/e23050581 - 8 May 2021

Cited by 5 | Viewed by 4450

Abstract

We focus on several questions arising during the modelling of quantum systems on a phase space. First, we discuss the choice of phase space and its structure. We include an interesting case of discrete phase space. Then, we introduce the respective algebras of [...] Read more.

We focus on several questions arising during the modelling of quantum systems on a phase space. First, we discuss the choice of phase space and its structure. We include an interesting case of discrete phase space. Then, we introduce the respective algebras of functions containing quantum observables. We also consider the possibility of performing strict calculations and indicate cases where only formal considerations can be performed. We analyse alternative realisations of strict and formal calculi, which are determined by different kernels. Finally, two classes of Wigner functions as representations of states are investigated. Full article

(This article belongs to the Special Issue Quantum Mechanics and Its Foundations)

22 pages, 637 KB

Open AccessEditor’s ChoiceArticle

Selecting an Effective Entropy Estimator for Short Sequences of Bits and Bytes with Maximum Entropy

by Lianet Contreras Rodríguez, Evaristo José Madarro-Capó , Carlos Miguel Legón-Pérez , Omar Rojas and Guillermo Sosa-Gómez

Entropy 2021, 23(5), 561; https://doi.org/10.3390/e23050561 - 30 Apr 2021

Cited by 17 | Viewed by 4727

Abstract

Entropy makes it possible to measure the uncertainty about an information source from the distribution of its output symbols. It is known that the maximum Shannon’s entropy of a discrete source of information is reached when its symbols follow a Uniform distribution. In [...] Read more.

Entropy makes it possible to measure the uncertainty about an information source from the distribution of its output symbols. It is known that the maximum Shannon’s entropy of a discrete source of information is reached when its symbols follow a Uniform distribution. In cryptography, these sources have great applications since they allow for the highest security standards to be reached. In this work, the most effective estimator is selected to estimate entropy in short samples of bytes and bits with maximum entropy. For this, 18 estimators were compared. Results concerning the comparisons published in the literature between these estimators are discussed. The most suitable estimator is determined experimentally, based on its bias, the mean square error short samples of bytes and bits. Full article

(This article belongs to the Special Issue Types of Entropies and Divergences with Their Applications)

► Show Figures

Figure 1

7 pages, 483 KB

Open AccessEditor’s ChoiceReview

Advances in Atomtronics

by Ron A. Pepino

Entropy 2021, 23(5), 534; https://doi.org/10.3390/e23050534 - 27 Apr 2021

Cited by 7 | Viewed by 3633

Abstract

Atomtronics is a relatively new subfield of atomic physics that aims to realize the device behavior of electronic components in ultracold atom-optical systems. The fact that these systems are coherent makes them particularly interesting since, in addition to current, one can impart quantum [...] Read more.

Atomtronics is a relatively new subfield of atomic physics that aims to realize the device behavior of electronic components in ultracold atom-optical systems. The fact that these systems are coherent makes them particularly interesting since, in addition to current, one can impart quantum states onto the current carriers themselves or perhaps perform quantum computational operations on them. After reviewing the fundamental ideas of this subfield, we report on the theoretical and experimental progress made towards developing externally-driven and closed loop devices. The functionality and potential applications for these atom analogs to electronic and spintronic systems is also discussed. Full article

(This article belongs to the Special Issue Noisy Intermediate-Scale Quantum Technologies (NISQ))

► Show Figures

Figure 1

19 pages, 1770 KB

Open AccessEditor’s ChoiceArticle

Unifying Large- and Small-Scale Theories of Coordination

by J. A. Scott Kelso

Entropy 2021, 23(5), 537; https://doi.org/10.3390/e23050537 - 27 Apr 2021

Cited by 52 | Viewed by 10239

Abstract

Coordination is a ubiquitous feature of all living things. It occurs by virtue of informational coupling among component parts and processes and can be quite specific (as when cells in the brain resonate to signals in the environment) or nonspecific (as when simple [...] Read more.

Coordination is a ubiquitous feature of all living things. It occurs by virtue of informational coupling among component parts and processes and can be quite specific (as when cells in the brain resonate to signals in the environment) or nonspecific (as when simple diffusion creates a source–sink dynamic for gene networks). Existing theoretical models of coordination—from bacteria to brains to social groups—typically focus on systems with very large numbers of elements (N→∞) or systems with only a few elements coupled together (typically N = 2). Though sharing a common inspiration in Nature’s propensity to generate dynamic patterns, both approaches have proceeded largely independent of each other. Ideally, one would like a theory that applies to phenomena observed on all scales. Recent experimental research by Mengsen Zhang and colleagues on intermediate-sized ensembles (in between the few and the many) proves to be the key to uniting large- and small-scale theories of coordination. Disorder–order transitions, multistability, order–order phase transitions, and especially metastability are shown to figure prominently on multiple levels of description, suggestive of a basic Coordination Dynamics that operates on all scales. This unified coordination dynamics turns out to be a marriage of two well-known models of large- and small-scale coordination: the former based on statistical mechanics (Kuramoto) and the latter based on the concepts of Synergetics and nonlinear dynamics (extended Haken–Kelso–Bunz or HKB). We show that models of the many and the few, previously quite unconnected, are thereby unified in a single formulation. The research has led to novel topological methods to handle the higher-dimensional dynamics of coordination in complex systems and has implications not only for understanding coordination but also for the design of (biorhythm inspired) computers. Full article

(This article belongs to the Special Issue Information and Self-Organization II)

► Show Figures

Figure 1

17 pages, 1122 KB

Open AccessEditor’s ChoiceArticle

α-Geodesical Skew Divergence

by Masanari Kimura and Hideitsu Hino

Entropy 2021, 23(5), 528; https://doi.org/10.3390/e23050528 - 25 Apr 2021

Cited by 5 | Viewed by 4298

Abstract

The asymmetric skew divergence smooths one of the distributions by mixing it, to a degree determined by the parameter

λ

, with the other distribution. Such divergence is an approximation of the KL divergence that does not require the target distribution to be [...] Read more.

The asymmetric skew divergence smooths one of the distributions by mixing it, to a degree determined by the parameter

λ

, with the other distribution. Such divergence is an approximation of the KL divergence that does not require the target distribution to be absolutely continuous with respect to the source distribution. In this paper, an information geometric generalization of the skew divergence called the

α

-geodesical skew divergence is proposed, and its properties are studied. Full article

► Show Figures

Figure 1

10 pages, 2520 KB

Open AccessEditor’s ChoiceArticle

Information-Efficient, Off-Center Sampling Results in Improved Precision in 3D Single-Particle Tracking Microscopy

by Chen Zhang and Kevin Welsher

Entropy 2021, 23(5), 498; https://doi.org/10.3390/e23050498 - 22 Apr 2021

Cited by 14 | Viewed by 4268

Abstract

In this work, we present a 3D single-particle tracking system that can apply tailored sampling patterns to selectively extract photons that yield the most information for particle localization. We demonstrate that off-center sampling at locations predicted by Fisher information utilizes photons most efficiently. [...] Read more.

In this work, we present a 3D single-particle tracking system that can apply tailored sampling patterns to selectively extract photons that yield the most information for particle localization. We demonstrate that off-center sampling at locations predicted by Fisher information utilizes photons most efficiently. When performing localization in a single dimension, optimized off-center sampling patterns gave doubled precision compared to uniform sampling. A ~20% increase in precision compared to uniform sampling can be achieved when a similar off-center pattern is used in 3D localization. Here, we systematically investigated the photon efficiency of different emission patterns in a diffraction-limited system and achieved higher precision than uniform sampling. The ability to maximize information from the limited number of photons demonstrated here is critical for particle tracking applications in biological samples, where photons may be limited. Full article

(This article belongs to the Special Issue Recent Advances in Single-Particle Tracking: Experiment and Analysis)

► Show Figures

Figure 1

20 pages, 879 KB

Open AccessEditor’s ChoiceArticle

Entanglement and Non-Locality in Quantum Protocols with Identical Particles

by Fabio Benatti, Roberto Floreanini and Ugo Marzolino

Entropy 2021, 23(4), 479; https://doi.org/10.3390/e23040479 - 18 Apr 2021

Cited by 5 | Viewed by 4889

Abstract

We study the role of entanglement and non-locality in quantum protocols that make use of systems of identical particles. Unlike in the case of distinguishable particles, the notions of entanglement and non-locality for systems whose constituents cannot be distinguished and singly addressed are [...] Read more.

We study the role of entanglement and non-locality in quantum protocols that make use of systems of identical particles. Unlike in the case of distinguishable particles, the notions of entanglement and non-locality for systems whose constituents cannot be distinguished and singly addressed are still debated. We clarify why the only approach that avoids incongruities and paradoxes is the one based on the second quantization formalism, whereby it is the entanglement of the modes that can be populated by the particles that really matters and not the particles themselves. Indeed, by means of a metrological and of a teleportation protocol, we show that inconsistencies arise in formulations that force entanglement and non-locality to be properties of the identical particles rather than of the modes they can occupy. The reason resides in the fact that orthogonal modes can always be addressed while identical particles cannot. Full article

(This article belongs to the Special Issue Quantum Information and Quantum Optics)

13 pages, 3015 KB

Open AccessEditor’s ChoiceArticle

Novel Features for Binary Time Series Based on Branch Length Similarity Entropy

by Sang-Hee Lee and Cheol-Min Park

Entropy 2021, 23(4), 480; https://doi.org/10.3390/e23040480 - 18 Apr 2021

Cited by 4 | Viewed by 3667

Abstract

Branch length similarity (BLS) entropy is defined in a network consisting of a single node and branches. In this study, we mapped the binary time-series signal to the circumference of the time circle so that the BLS entropy can be calculated for the [...] Read more.

Branch length similarity (BLS) entropy is defined in a network consisting of a single node and branches. In this study, we mapped the binary time-series signal to the circumference of the time circle so that the BLS entropy can be calculated for the binary time-series. We obtained the BLS entropy values for “1” signals on the time circle. The set of values are the BLS entropy profile. We selected the local maximum (minimum) point, slope, and inflection point of the entropy profile as the characteristic features of the binary time-series and investigated and explored their significance. The local maximum (minimum) point indicates the time at which the rate of change in the signal density becomes zero. The slope and inflection points correspond to the degree of change in the signal density and the time at which the signal density changes occur, respectively. Moreover, we show that the characteristic features can be widely used in binary time-series analysis by characterizing the movement trajectory of Caenorhabditis elegans. We also mention the problems that need to be explored mathematically in relation to the features and propose candidates for additional features based on the BLS entropy profile. Full article

(This article belongs to the Special Issue Information Entropy Algorithms for Image, Video, and Signal Processing)

► Show Figures

Figure 1

17 pages, 3153 KB

Open AccessEditor’s ChoiceArticle

Low-Frequency Seismic Noise Properties in the Japanese Islands

by Alexey Lyubushin

Entropy 2021, 23(4), 474; https://doi.org/10.3390/e23040474 - 16 Apr 2021

Cited by 17 | Viewed by 6769

Abstract

The records of seismic noise in Japan for the period of 1997–2020, which includes the Tohoku seismic catastrophe on 11 March 2011, are considered. The following properties of noise are analyzed: The wavelet-based Donoho–Johnston index, the singularity spectrum support width, and the entropy [...] Read more.

The records of seismic noise in Japan for the period of 1997–2020, which includes the Tohoku seismic catastrophe on 11 March 2011, are considered. The following properties of noise are analyzed: The wavelet-based Donoho–Johnston index, the singularity spectrum support width, and the entropy of the wavelet coefficients. The question of whether precursors of strong earthquakes can be formulated on their basis is investigated. Attention is paid to the time interval after the Tohoku mega-earthquake to the trends in the mean properties of low-frequency seismic noise, which reflect the constant simplification of the statistical structure of seismic vibrations. Estimates of two-dimensional probability densities of extreme values are presented, which highlight the places in which extreme values of seismic noise properties are most often realized. The estimates of the probability densities of extreme values coincide with each other and have a maximum in the region:

30 ° N \leq Lat \leq 34 ° N

,

136 ° E \leq Lon \leq 140 ° E

. The main conclusions of the conducted studies are that the preparation of a strong earthquake is accompanied by a simplification of the structure of seismic noise. It is shown that bursts of coherence between the time series of the day length and the noise properties within annual time window precede bursts of released seismic energy. The value of the lag in the release of seismic energy relative to bursts of coherence is about 1.5 years, which can be used to declare a time interval of high seismic hazard after reaching the peak of coherence. Full article

(This article belongs to the Special Issue Complex Systems Time Series Analysis and Modeling for Geoscience)

► Show Figures

Figure 1

28 pages, 1106 KB

Open AccessEditor’s ChoiceArticle

On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius

by Frank Nielsen

Entropy 2021, 23(4), 464; https://doi.org/10.3390/e23040464 - 14 Apr 2021

Cited by 35 | Viewed by 13065

Abstract

We generalize the Jensen-Shannon divergence and the Jensen-Shannon diversity index by considering a variational definition with respect to a generic mean, thereby extending the notion of Sibson’s information radius. The variational definition applies to any arbitrary distance and yields a new way to [...] Read more.

We generalize the Jensen-Shannon divergence and the Jensen-Shannon diversity index by considering a variational definition with respect to a generic mean, thereby extending the notion of Sibson’s information radius. The variational definition applies to any arbitrary distance and yields a new way to define a Jensen-Shannon symmetrization of distances. When the variational optimization is further constrained to belong to prescribed families of probability measures, we get relative Jensen-Shannon divergences and their equivalent Jensen-Shannon symmetrizations of distances that generalize the concept of information projections. Finally, we touch upon applications of these variational Jensen-Shannon divergences and diversity indices to clustering and quantization tasks of probability measures, including statistical mixtures. Full article

(This article belongs to the Special Issue Selected Papers from the 5th conference on Geometric Science of Information)

► Show Figures

Graphical abstract

14 pages, 1344 KB

Open AccessEditor’s ChoiceArticle

Federated Quantum Machine Learning

by Samuel Yen-Chi Chen and Shinjae Yoo

Entropy 2021, 23(4), 460; https://doi.org/10.3390/e23040460 - 13 Apr 2021

Cited by 159 | Viewed by 15667

Abstract

Distributed training across several quantum computers could significantly improve the training time and if we could share the learned model, not the data, it could potentially improve the data privacy as the training would happen where the data is located. One of the [...] Read more.

Distributed training across several quantum computers could significantly improve the training time and if we could share the learned model, not the data, it could potentially improve the data privacy as the training would happen where the data is located. One of the potential schemes to achieve this property is the federated learning (FL), which consists of several clients or local nodes learning on their own data and a central node to aggregate the models collected from those local nodes. However, to the best of our knowledge, no work has been done in quantum machine learning (QML) in federation setting yet. In this work, we present the federated training on hybrid quantum-classical machine learning models although our framework could be generalized to pure quantum machine learning model. Specifically, we consider the quantum neural network (QNN) coupled with classical pre-trained convolutional model. Our distributed federated learning scheme demonstrated almost the same level of trained model accuracies and yet significantly faster distributed training. It demonstrates a promising future research direction for scaling and privacy aspects. Full article

(This article belongs to the Special Issue Noisy Intermediate-Scale Quantum Technologies (NISQ))

► Show Figures

Figure 1

40 pages, 1007 KB

Open AccessEditor’s ChoiceArticle

Phase Transitions in Transfer Learning for High-Dimensional Perceptrons

by Oussama Dhifallah and Yue M. Lu

Entropy 2021, 23(4), 400; https://doi.org/10.3390/e23040400 - 27 Mar 2021

Cited by 17 | Viewed by 5431

Abstract

Transfer learning seeks to improve the generalization performance of a target task by exploiting the knowledge learned from a related source task. Central questions include deciding what information one should transfer and when transfer can be beneficial. The latter question is related to [...] Read more.

Transfer learning seeks to improve the generalization performance of a target task by exploiting the knowledge learned from a related source task. Central questions include deciding what information one should transfer and when transfer can be beneficial. The latter question is related to the so-called negative transfer phenomenon, where the transferred source information actually reduces the generalization performance of the target task. This happens when the two tasks are sufficiently dissimilar. In this paper, we present a theoretical analysis of transfer learning by studying a pair of related perceptron learning tasks. Despite the simplicity of our model, it reproduces several key phenomena observed in practice. Specifically, our asymptotic analysis reveals a phase transition from negative transfer to positive transfer as the similarity of the two tasks moves past a well-defined threshold. Full article

(This article belongs to the Special Issue The Role of Signal Processing and Information Theory in Modern Machine Learning)

► Show Figures

Figure 1

14 pages, 4567 KB

Open AccessEditor’s ChoiceArticle

Interpretable Multi-Head Self-Attention Architecture for Sarcasm Detection in Social Media

by Ramya Akula and Ivan Garibay

Entropy 2021, 23(4), 394; https://doi.org/10.3390/e23040394 - 26 Mar 2021

Cited by 55 | Viewed by 22911

Abstract

With the online presence of more than half the world population, social media plays a very important role in the lives of individuals as well as businesses alike. Social media enables businesses to advertise their products, build brand value, and reach out to [...] Read more.

With the online presence of more than half the world population, social media plays a very important role in the lives of individuals as well as businesses alike. Social media enables businesses to advertise their products, build brand value, and reach out to their customers. To leverage these social media platforms, it is important for businesses to process customer feedback in the form of posts and tweets. Sentiment analysis is the process of identifying the emotion, either positive, negative or neutral, associated with these social media texts. The presence of sarcasm in texts is the main hindrance in the performance of sentiment analysis. Sarcasm is a linguistic expression often used to communicate the opposite of what is said, usually something that is very unpleasant, with an intention to insult or ridicule. Inherent ambiguity in sarcastic expressions make sarcasm detection very difficult. In this work, we focus on detecting sarcasm in textual conversations from various social networking platforms and online media. To this end, we develop an interpretable deep learning model using multi-head self-attention and gated recurrent units. The multi-head self-attention module aids in identifying crucial sarcastic cue-words from the input, and the recurrent units learn long-range dependencies between these cue-words to better classify the input text. We show the effectiveness of our approach by achieving state-of-the-art results on multiple datasets from social networking platforms and online media. Models trained using our proposed approach are easily interpretable and enable identifying sarcastic cues in the input text which contribute to the final classification score. We visualize the learned attention weights on a few sample input texts to showcase the effectiveness and interpretability of our model. Full article

► Show Figures

Figure 1

17 pages, 1936 KB

Open AccessEditor’s ChoiceArticle

Causality and Information Transfer Between the Solar Wind and the Magnetosphere–Ionosphere System

by Pouya Manshour, Georgios Balasis, Giuseppe Consolini, Constantinos Papadimitriou and Milan Paluš

Entropy 2021, 23(4), 390; https://doi.org/10.3390/e23040390 - 25 Mar 2021

Cited by 36 | Viewed by 5283

Abstract

An information-theoretic approach for detecting causality and information transfer is used to identify interactions of solar activity and interplanetary medium conditions with the Earth’s magnetosphere–ionosphere systems. A causal information transfer from the solar wind parameters to geomagnetic indices is detected. The vertical component [...] Read more.

An information-theoretic approach for detecting causality and information transfer is used to identify interactions of solar activity and interplanetary medium conditions with the Earth’s magnetosphere–ionosphere systems. A causal information transfer from the solar wind parameters to geomagnetic indices is detected. The vertical component of the interplanetary magnetic field (

B z

) influences the auroral electrojet (AE) index with an information transfer delay of 10 min and the geomagnetic disturbances at mid-latitudes measured by the symmetric field in the H component (SYM-H) index with a delay of about 30 min. Using a properly conditioned causality measure, no causal link between AE and SYM-H, or between magnetospheric substorms and magnetic storms can be detected. The observed causal relations can be described as linear time-delayed information transfer. Full article

(This article belongs to the Special Issue Information Transfer, Entropy Production, Irreversibility and Time Series Analysis)

► Show Figures

Graphical abstract

14 pages, 326 KB

Open AccessEditor’s ChoiceArticle

On the Classical Capacity of General Quantum Gaussian Measurement

by Alexander Holevo

Entropy 2021, 23(3), 377; https://doi.org/10.3390/e23030377 - 21 Mar 2021

Cited by 14 | Viewed by 4306

Abstract

In this paper, we consider the classical capacity problem for Gaussian measurement channels. We establish Gaussianity of the average state of the optimal ensemble in the general case and discuss the Hypothesis of Gaussian Maximizers concerning the structure of the ensemble. Then, we [...] Read more.

In this paper, we consider the classical capacity problem for Gaussian measurement channels. We establish Gaussianity of the average state of the optimal ensemble in the general case and discuss the Hypothesis of Gaussian Maximizers concerning the structure of the ensemble. Then, we consider the case of one mode in detail, including the dual problem of accessible information of a Gaussian ensemble. Our findings are relevant to practical situations in quantum communications where the receiver is Gaussian (say, a general-dyne detection) and concatenation of the Gaussian channel and the receiver can be considered as one Gaussian measurement channel. Our efforts in this and preceding papers are then aimed at establishing full Gaussianity of the optimal ensemble (usually taken as an assumption) in such schemes. Full article

(This article belongs to the Special Issue Quantum Communication, Quantum Radar, and Quantum Cipher)

► Show Figures

Figure 1

11 pages, 1340 KB

Open AccessEditor’s ChoiceArticle

Spectral Ranking of Causal Influence in Complex Systems

by Errol Zalmijn, Tom Heskes and Tom Claassen

Entropy 2021, 23(3), 369; https://doi.org/10.3390/e23030369 - 20 Mar 2021

Cited by 1 | Viewed by 5793

Abstract

Similar to natural complex systems, such as the Earth’s climate or a living cell, semiconductor lithography systems are characterized by nonlinear dynamics across more than a dozen orders of magnitude in space and time. Thousands of sensors measure relevant process variables at appropriate [...] Read more.

Similar to natural complex systems, such as the Earth’s climate or a living cell, semiconductor lithography systems are characterized by nonlinear dynamics across more than a dozen orders of magnitude in space and time. Thousands of sensors measure relevant process variables at appropriate sampling rates, to provide time series as primary sources for system diagnostics. However, high-dimensionality, non-linearity and non-stationarity of the data are major challenges to efficiently, yet accurately, diagnose rare or new system issues by merely using model-based approaches. To reliably narrow down the causal search space, we validate a ranking algorithm that applies transfer entropy for bivariate interaction analysis of a system’s multivariate time series to obtain a weighted directed graph, and graph eigenvector centrality to identify the system’s most important sources of original information or causal influence. The results suggest that this approach robustly identifies the true drivers or causes of a complex system’s deviant behavior, even when its reconstructed information transfer network includes redundant edges. Full article

► Show Figures

Figure 1

18 pages, 3275 KB

Open AccessEditor’s ChoiceArticle

Stirling Refrigerating Machine Modeling Using Schmidt and Finite Physical Dimensions Thermodynamic Models: A Comparison with Experiments

by Cătălina Dobre, Lavinia Grosu, Alexandru Dobrovicescu, Georgiana Chişiu and Mihaela Constantin

Entropy 2021, 23(3), 368; https://doi.org/10.3390/e23030368 - 19 Mar 2021

Cited by 9 | Viewed by 6227

Abstract

The purpose of the study is to show that two simple models that take into account only the irreversibility due to temperature difference in the heat exchangers and imperfect regeneration are able to indicate refrigerating machine behavior. In the present paper, the finite [...] Read more.

The purpose of the study is to show that two simple models that take into account only the irreversibility due to temperature difference in the heat exchangers and imperfect regeneration are able to indicate refrigerating machine behavior. In the present paper, the finite physical dimensions thermodynamics (FPDT) method and 0-D modeling using the Schmidt model with imperfect regeneration were applied in the study of a β type Stirling refrigeration machine.The 0-D modeling is improved by including the irreversibility caused by imperfect regeneration and the finite temperature difference between the gas and the heat exchangers wall. A flowchart of the Stirling refrigerator exergy balance is presented to show the internal and external irreversibilities. It is found that the irreversibility at the regenerator level is more important than that at the heat exchangers level. The energies exchanged by the working gas are expressed according to the practical parameters, necessary for the engineer during the entire project. The results of the two thermodynamic models are presented in comparison with the experimental results, which leads to validation of the proposed FPDT model for the functional and constructive parameters of the studied refrigerating machine. Full article

(This article belongs to the Special Issue Carnot Cycle and Heat Engine Fundamentals and Applications II)

► Show Figures

Figure 1

29 pages, 1389 KB

Open AccessEditor’s ChoiceArticle

Mechanism Integrated Information

by Leonardo S. Barbosa, William Marshall, Larissa Albantakis and Giulio Tononi

Entropy 2021, 23(3), 362; https://doi.org/10.3390/e23030362 - 18 Mar 2021

Cited by 44 | Viewed by 9203

Abstract

The Integrated Information Theory (IIT) of consciousness starts from essential phenomenological properties, which are then translated into postulates that any physical system must satisfy in order to specify the physical substrate of consciousness. We recently introduced an information measure (Barbosa et al., 2020) [...] Read more.

The Integrated Information Theory (IIT) of consciousness starts from essential phenomenological properties, which are then translated into postulates that any physical system must satisfy in order to specify the physical substrate of consciousness. We recently introduced an information measure (Barbosa et al., 2020) that captures three postulates of IIT—existence, intrinsicality and information—and is unique. Here we show that the new measure also satisfies the remaining postulates of IIT—integration and exclusion—and create the framework that identifies maximally irreducible mechanisms. These mechanisms can then form maximally irreducible systems, which in turn will specify the physical substrate of conscious experience. Full article

(This article belongs to the Special Issue Integrated Information Theory and Consciousness)

► Show Figures

Graphical abstract

14 pages, 325 KB

Open AccessEditor’s ChoiceArticle

Critical Comparison of MaxCal and Other Stochastic Modeling Approaches in Analysis of Gene Networks

by Taylor Firman, Jonathan Huihui, Austin R. Clark and Kingshuk Ghosh

Entropy 2021, 23(3), 357; https://doi.org/10.3390/e23030357 - 17 Mar 2021

Cited by 1 | Viewed by 3943

Abstract

Learning the underlying details of a gene network with feedback is critical in designing new synthetic circuits. Yet, quantitative characterization of these circuits remains limited. This is due to the fact that experiments can only measure partial information from which the details of [...] Read more.

Learning the underlying details of a gene network with feedback is critical in designing new synthetic circuits. Yet, quantitative characterization of these circuits remains limited. This is due to the fact that experiments can only measure partial information from which the details of the circuit must be inferred. One potentially useful avenue is to harness hidden information from single-cell stochastic gene expression time trajectories measured for long periods of time—recorded at frequent intervals—over multiple cells. This raises the feasibility vs. accuracy dilemma while deciding between different models of mining these stochastic trajectories. We demonstrate that inference based on the Maximum Caliber (MaxCal) principle is the method of choice by critically evaluating its computational efficiency and accuracy against two other typical modeling approaches: (i) a detailed model (DM) with explicit consideration of multiple molecules including protein-promoter interaction, and (ii) a coarse-grain model (CGM) using Hill type functions to model feedback. MaxCal provides a reasonably accurate model while being significantly more computationally efficient than DM and CGM. Furthermore, MaxCal requires minimal assumptions since it is a top-down approach and allows systematic model improvement by including constraints of higher order, in contrast to traditional bottom-up approaches that require more parameters or ad hoc assumptions. Thus, based on efficiency, accuracy, and ability to build minimal models, we propose MaxCal as a superior alternative to traditional approaches (DM, CGM) when inferring underlying details of gene circuits with feedback from limited data. Full article

(This article belongs to the Special Issue Information Theory and Biology: Seeking General Principles)

16 pages, 5950 KB

Open AccessEditor’s ChoiceReview

Beating Standard Quantum Limit with Weak Measurement

by Geng Chen, Peng Yin, Wen-Hao Zhang, Gong-Chu Li, Chuan-Feng Li and Guang-Can Guo

Entropy 2021, 23(3), 354; https://doi.org/10.3390/e23030354 - 16 Mar 2021

Cited by 19 | Viewed by 7214

Abstract

Weak measurements have been under intensive investigation in both experiment and theory. Numerous experiments have indicated that the amplified meter shift is produced by the post-selection, yielding an improved precision compared to conventional methods. However, this amplification effect comes at the cost of [...] Read more.

Weak measurements have been under intensive investigation in both experiment and theory. Numerous experiments have indicated that the amplified meter shift is produced by the post-selection, yielding an improved precision compared to conventional methods. However, this amplification effect comes at the cost of a reduced rate of acquiring data, which leads to an increasing uncertainty to determine the level of meter shift. From this point of view, a number of theoretical works have suggested that weak measurements cannot improve the precision, or even damage the metrology information due to the post-selection. In this review, we give a comprehensive analysis of the weak measurements to justify their positive effect on prompting measurement precision. As a further step, we introduce two modified weak measurement protocols to boost the precision beyond the standard quantum limit. Compared to previous works beating the standard quantum limit, these protocols are free of using entangled or squeezed states. The achieved precision outperforms that of the conventional method by two orders of magnitude and attains a practical Heisenberg scaling up to

n = 10^{6}

photons. Full article

(This article belongs to the Special Issue Lectures on Recent Experimental Achievements in Quantum-Enhanced Technologies)

► Show Figures

Figure 1

29 pages, 535 KB

Open AccessEditor’s ChoiceArticle

Crowded Trades, Market Clustering, and Price Instability

by Marc van Kralingen, Diego Garlaschelli, Karolina Scholtus and Iman van Lelyveld

Entropy 2021, 23(3), 336; https://doi.org/10.3390/e23030336 - 12 Mar 2021

Cited by 7 | Viewed by 8311

Abstract

Crowded trades by similarly trading peers influence the dynamics of asset prices, possibly creating systemic risk. We propose a market clustering measure using granular trading data. For each stock, the clustering measure captures the degree of trading overlap among any two investors in [...] Read more.

Crowded trades by similarly trading peers influence the dynamics of asset prices, possibly creating systemic risk. We propose a market clustering measure using granular trading data. For each stock, the clustering measure captures the degree of trading overlap among any two investors in that stock, based on a comparison with the expected crowding in a null model where trades are maximally random while still respecting the empirical heterogeneity of both stocks and investors. We investigate the effect of crowded trades on stock price stability and present evidence that market clustering has a causal effect on the properties of the tails of the stock return distribution, particularly the positive tail, even after controlling for commonly considered risk drivers. Reduced investor pool diversity could thus negatively affect stock price stability. Full article

(This article belongs to the Special Issue Entropy-Based Applications in Economics, Finance, and Management)

► Show Figures

Figure 1

8 pages, 1014 KB

Open AccessEditor’s ChoiceArticle

Quantum Speed Limit and Divisibility of the Dynamical Map

by Jose Teittinen and Sabrina Maniscalco

Entropy 2021, 23(3), 331; https://doi.org/10.3390/e23030331 - 11 Mar 2021

Cited by 16 | Viewed by 3996

Abstract

The quantum speed limit (QSL) is the theoretical lower limit of the time for a quantum system to evolve from a given state to another one. Interestingly, it has been shown that non-Markovianity can be used to speed-up the dynamics and to lower [...] Read more.

The quantum speed limit (QSL) is the theoretical lower limit of the time for a quantum system to evolve from a given state to another one. Interestingly, it has been shown that non-Markovianity can be used to speed-up the dynamics and to lower the QSL time, although this behaviour is not universal. In this paper, we further carry on the investigation on the connection between QSL and non-Markovianity by looking at the effects of P- and CP-divisibility of the dynamical map to the quantum speed limit. We show that the speed-up can also be observed under P- and CP-divisible dynamics, and that the speed-up is not necessarily tied to the transition from P-divisible to non-P-divisible dynamics. Full article

(This article belongs to the Special Issue Quantum Information Concepts in Open Quantum Systems)

► Show Figures

Figure 1

19 pages, 362 KB

Open AccessEditor’s ChoiceReview

Information Theory for Agents in Artificial Intelligence, Psychology, and Economics

by Michael S. Harré

Entropy 2021, 23(3), 310; https://doi.org/10.3390/e23030310 - 6 Mar 2021

Cited by 20 | Viewed by 9612

Abstract

This review looks at some of the central relationships between artificial intelligence, psychology, and economics through the lens of information theory, specifically focusing on formal models of decision-theory. In doing so we look at a particular approach that each field has adopted and [...] Read more.

This review looks at some of the central relationships between artificial intelligence, psychology, and economics through the lens of information theory, specifically focusing on formal models of decision-theory. In doing so we look at a particular approach that each field has adopted and how information theory has informed the development of the ideas of each field. A key theme is expected utility theory, its connection to information theory, the Bayesian approach to decision-making and forms of (bounded) rationality. What emerges from this review is a broadly unified formal perspective derived from three very different starting points that reflect the unique principles of each field. Each of the three approaches reviewed can, in principle at least, be implemented in a computational model in such a way that, with sufficient computational power, they could be compared with human abilities in complex tasks. However, a central critique that can be applied to all three approaches was first put forward by Savage in The Foundations of Statistics and recently brought to the fore by the economist Binmore: Bayesian approaches to decision-making work in what Savage called ‘small worlds’ but cannot work in ‘large worlds’. This point, in various different guises, is central to some of the current debates about the power of artificial intelligence and its relationship to human-like learning and decision-making. Recent work on artificial intelligence has gone some way to bridging this gap but significant questions remain to be answered in all three fields in order to make progress in producing realistic models of human decision-making in the real world in which we live in. Full article

(This article belongs to the Special Issue Information Theory at the Crossroads of Artificial Intelligence, Human Cognition, and Economics)

► Show Figures

Graphical abstract

32 pages, 4858 KB

Open AccessEditor’s ChoiceArticle

Why Do Big Data and Machine Learning Entail the Fractional Dynamics?

by Haoyu Niu, YangQuan Chen and Bruce J. West

Entropy 2021, 23(3), 297; https://doi.org/10.3390/e23030297 - 28 Feb 2021

Cited by 30 | Viewed by 9013

Abstract

Fractional-order calculus is about the differentiation and integration of non-integer orders. Fractional calculus (FC) is based on fractional-order thinking (FOT) and has been shown to help us to understand complex systems better, improve the processing of complex signals, enhance the control of complex [...] Read more.

Fractional-order calculus is about the differentiation and integration of non-integer orders. Fractional calculus (FC) is based on fractional-order thinking (FOT) and has been shown to help us to understand complex systems better, improve the processing of complex signals, enhance the control of complex systems, increase the performance of optimization, and even extend the enabling of the potential for creativity. In this article, the authors discuss the fractional dynamics, FOT and rich fractional stochastic models. First, the use of fractional dynamics in big data analytics for quantifying big data variability stemming from the generation of complex systems is justified. Second, we show why fractional dynamics is needed in machine learning and optimal randomness when asking: “is there a more optimal way to optimize?”. Third, an optimal randomness case study for a stochastic configuration network (SCN) machine-learning method with heavy-tailed distributions is discussed. Finally, views on big data and (physics-informed) machine learning with fractional dynamics for future research are presented with concluding remarks. Full article

(This article belongs to the Special Issue Fractional Calculus and the Future of Science)

► Show Figures

Figure 1

19 pages, 5707 KB

Open AccessEditor’s ChoiceArticle

Non-Extensive Statistical Analysis of Acoustic Emissions: The Variability of Entropic Index q during Loading of Brittle Materials Until Fracture

by Andronikos Loukidis, Dimos Triantis and Ilias Stavrakas

Entropy 2021, 23(3), 276; https://doi.org/10.3390/e23030276 - 25 Feb 2021

Cited by 7 | Viewed by 2870

Abstract

Non-extensive statistical mechanics (NESM), introduced by Tsallis based on the principle of non-additive entropy, is a generalisation of the Boltzmann–Gibbs statistics. NESM has been shown to provide the necessary theoretical and analytical implementation for studying complex systems such as the fracture mechanisms and [...] Read more.

Non-extensive statistical mechanics (NESM), introduced by Tsallis based on the principle of non-additive entropy, is a generalisation of the Boltzmann–Gibbs statistics. NESM has been shown to provide the necessary theoretical and analytical implementation for studying complex systems such as the fracture mechanisms and crack evolution processes that occur in mechanically loaded specimens of brittle materials. In the current work, acoustic emission (AE) data recorded when marble and cement mortar specimens were subjected to three distinct loading protocols until fracture, are discussed in the context of NESM. The NESM analysis showed that the cumulative distribution functions of the AE interevent times (i.e., the time interval between successive AE hits) follow a q-exponential function. For each examined specimen, the corresponding Tsallis entropic q-indices and the parameters β_q and

τ_{q}

were calculated. The entropic index

q

shows a systematic behaviour strongly related to the various stages of the implemented loading protocols for all the examined specimens. Results seem to support the idea of using the entropic index

q

as a potential pre-failure indicator for the impending catastrophic fracture of the mechanically loaded specimens. Full article

(This article belongs to the Special Issue Complex Systems Time Series Analysis and Modeling for Geoscience)

► Show Figures

Figure 1

33 pages, 477 KB

Open AccessEditor’s ChoiceArticle

Trade-offs between Error Exponents and Excess-Rate Exponents of Typical Slepian–Wolf Codes

by Ran Tamir (Averbuch) and Neri Merhav

Entropy 2021, 23(3), 265; https://doi.org/10.3390/e23030265 - 24 Feb 2021

Cited by 7 | Viewed by 2803

Abstract

Typical random codes (TRCs) in a communication scenario of source coding with side information in the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random binning code ensemble. In [...] Read more.

Typical random codes (TRCs) in a communication scenario of source coding with side information in the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random binning code ensemble. In this code ensemble, the relatively small type classes of the source are deterministically partitioned into the available bins in a one-to-one manner. As a consequence, the error probability decreases dramatically. The random binning error exponent and the error exponent of the TRCs are derived and proved to be equal to one another in a few important special cases. We show that the performance under optimal decoding can be attained also by certain universal decoders, e.g., the stochastic likelihood decoder with an empirical entropy metric. Moreover, we discuss the trade-offs between the error exponent and the excess-rate exponent for the typical random semi-deterministic code and characterize its optimal rate function. We show that for any pair of correlated information sources, both error and excess-rate probabilities exponential vanish when the blocklength tends to infinity. Full article

(This article belongs to the Special Issue Multiuser Information Theory III)

► Show Figures

Figure 1

30 pages, 628 KB

Open AccessEditor’s ChoiceArticle

Lapsing Quickly into Fatalism: Bell on Backward Causation

by Travis Norsen and Huw Price

Entropy 2021, 23(2), 251; https://doi.org/10.3390/e23020251 - 22 Feb 2021

Cited by 3 | Viewed by 6090

Abstract

This is a dialogue between Huw Price and Travis Norsen, loosely inspired by a letter that Price received from J. S. Bell in 1988. The main topic of discussion is Bell’s views about retrocausal approaches to quantum theory and their relevance to contemporary [...] Read more.

This is a dialogue between Huw Price and Travis Norsen, loosely inspired by a letter that Price received from J. S. Bell in 1988. The main topic of discussion is Bell’s views about retrocausal approaches to quantum theory and their relevance to contemporary issues. Full article

(This article belongs to the Special Issue Quantum Theory and Causation)

► Show Figures

Figure 1

14 pages, 1815 KB

Open AccessEditor’s ChoiceArticle

Signature of Generalized Gibbs Ensemble Deviation from Equilibrium: Negative Absorption Induced by a Local Quench

by Lorenzo Rossi, Fabrizio Dolcini, Fabio Cavaliere, Niccolò Traverso Ziani, Maura Sassetti and Fausto Rossi

Entropy 2021, 23(2), 220; https://doi.org/10.3390/e23020220 - 11 Feb 2021

Cited by 7 | Viewed by 3216

Abstract

When a parameter quench is performed in an isolated quantum system with a complete set of constants of motion, its out of equilibrium dynamics is considered to be well captured by the Generalized Gibbs Ensemble (GGE), characterized by a set [...] Read more.

When a parameter quench is performed in an isolated quantum system with a complete set of constants of motion, its out of equilibrium dynamics is considered to be well captured by the Generalized Gibbs Ensemble (GGE), characterized by a set

{λ_{α}}

of coefficients related to the constants of motion. We determine the most elementary GGE deviation from the equilibrium distribution that leads to detectable effects. By quenching a suitable local attractive potential in a one-dimensional electron system, the resulting GGE differs from equilibrium by only one single

λ_{α}

, corresponding to the emergence of an only partially occupied bound state lying below a fully occupied continuum of states. The effect is shown to induce optical gain, i.e., a negative peak in the absorption spectrum, indicating the stimulated emission of radiation, enabling one to identify GGE signatures in fermionic systems through optical measurements. We discuss the implementation in realistic setups. Full article

(This article belongs to the Section Non-equilibrium Phenomena)

► Show Figures

Figure 1

13 pages, 363 KB

Open AccessEditor’s ChoiceArticle

Selection of Embedding Dimension and Delay Time in Phase Space Reconstruction via Symbolic Dynamics

by Mariano Matilla-García, Isidro Morales, Jose Miguel Rodríguez and Manuel Ruiz Marín

Entropy 2021, 23(2), 221; https://doi.org/10.3390/e23020221 - 11 Feb 2021

Cited by 49 | Viewed by 8387

Abstract

The modeling and prediction of chaotic time series require proper reconstruction of the state space from the available data in order to successfully estimate invariant properties of the embedded attractor. Thus, one must choose appropriate time delay

τ^{*}

and embedding dimension p [...] Read more.

The modeling and prediction of chaotic time series require proper reconstruction of the state space from the available data in order to successfully estimate invariant properties of the embedded attractor. Thus, one must choose appropriate time delay

τ^{*}

and embedding dimension p for phase space reconstruction. The value of

τ^{*}

can be estimated from the Mutual Information, but this method is rather cumbersome computationally. Additionally, some researchers have recommended that

τ^{*}

should be chosen to be dependent on the embedding dimension p by means of an appropriate value for the time delay

τ_{w} = (p - 1) τ^{*}

, which is the optimal time delay for independence of the time series. The C-C method, based on Correlation Integral, is a method simpler than Mutual Information and has been proposed to select optimally

τ_{w}

and

τ^{*}

. In this paper, we suggest a simple method for estimating

τ^{*}

and

τ_{w}

based on symbolic analysis and symbolic entropy. As in the C-C method,

τ^{*}

is estimated as the first local optimal time delay and

τ_{w}

as the time delay for independence of the time series. The method is applied to several chaotic time series that are the base of comparison for several techniques. The numerical simulations for these systems verify that the proposed symbolic-based method is useful for practitioners and, according to the studied models, has a better performance than the C-C method for the choice of the time delay and embedding dimension. In addition, the method is applied to EEG data in order to study and compare some dynamic characteristics of brain activity under epileptic episodes Full article

(This article belongs to the Special Issue Information theory and Symbolic Analysis: Theory and Applications)

► Show Figures

Figure 1

35 pages, 1675 KB

Open AccessEditor’s ChoiceReview

The Entropy Universe

by Maria Ribeiro, Teresa Henriques, Luísa Castro, André Souto, Luís Antunes, Cristina Costa-Santos and Andreia Teixeira

Entropy 2021, 23(2), 222; https://doi.org/10.3390/e23020222 - 11 Feb 2021

Cited by 91 | Viewed by 19104

Abstract

About 160 years ago, the concept of entropy was introduced in thermodynamics by Rudolf Clausius. Since then, it has been continually extended, interpreted, and applied by researchers in many scientific fields, such as general physics, information theory, chaos theory, data mining, and mathematical [...] Read more.

About 160 years ago, the concept of entropy was introduced in thermodynamics by Rudolf Clausius. Since then, it has been continually extended, interpreted, and applied by researchers in many scientific fields, such as general physics, information theory, chaos theory, data mining, and mathematical linguistics. This paper presents The Entropy Universe, which aims to review the many variants of entropies applied to time-series. The purpose is to answer research questions such as: How did each entropy emerge? What is the mathematical definition of each variant of entropy? How are entropies related to each other? What are the most applied scientific fields for each entropy? We describe in-depth the relationship between the most applied entropies in time-series for different scientific fields, establishing bases for researchers to properly choose the variant of entropy most suitable for their data. The number of citations over the past sixteen years of each paper proposing a new entropy was also accessed. The Shannon/differential, the Tsallis, the sample, the permutation, and the approximate entropies were the most cited ones. Based on the ten research areas with the most significant number of records obtained in the Web of Science and Scopus, the areas in which the entropies are more applied are computer science, physics, mathematics, and engineering. The universe of entropies is growing each day, either due to the introducing new variants either due to novel applications. Knowing each entropy’s strengths and of limitations is essential to ensure the proper improvement of this research field. Full article

(This article belongs to the Special Issue Review Papers for Entropy)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Editor’s Choice Articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI