Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.7
2.7
Journals
Entropy
Special Issues
Information Flow in Neural Systems
entropy-logo
Submit to Entropy Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Information Flow in Neural Systems

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (10 May 2021) | Viewed by 9032

Special Issue Editors

Prof. Pulkit Grover

E-Mail Website
Guest Editor
Department of Electrical & Computer Engineering, Biomedical Engineering, and Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA
Interests: information theory; theory of computation; neuroscience; neuroengineering; dynamical systems
Prof. Christopher Quinn

E-Mail Website
Guest Editor
Department of Computer Science, Iowa State University, Ames, IA, USA
Interests: information theory; statistical signal processing; machine learning; neuroscience
Dr. Gabriel Schamberg

E-Mail
Guest Editor
Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Interests: causality; directed information theory; optimal control; statistical learning; neural signal processing

Special Issue Information

Dear Colleagues,

In an exciting confluence, information flows in neural networks—both artificial and natural—are garnering immense interest. While the term information flow is used frequently in practical contexts, such as clinical neuroscience or the optimization/interpretation of artificial neural networks, fundamental exploration of the topic has received limited attention. Societal implications of defining, understanding, designing, and/or affecting information flows are deep and broad, influencing all aspects of our lives. Many of these issues require careful and rigorous approaches that have only recently begun being developed.

This Special Issue focuses on core information theoretic issues pertaining to flows of information in natural and artificial neural networks. Information theory here is to be interpreted broadly, including, for instance, classical (Shannon) information theory, algorithmic information theory, control theory, and integrated information theory. The issue is intended to have a balanced representation between natural and artificial worlds, and papers connecting the two, or critiquing the perceived connection between the two, are also of interest. The Special Issue solicits papers that are, in their essence, intellectual and/or theoretical, although demonstration on real or synthetic datasets is encouraged when possible.
This Special Issue will assimilate the current approaches to the following (and related) topics:

  • Axiomatic definitions, measures and/or estimators of information flow in neural systems;
  • Models for controlling information flow;
  • Scalability of information flow methods in high dimensional neural systems;
  • Analysis and extensions of established information flow measures;
  • Models of information flow in clinical neuroscientific settings;
  • Connections between information flow and causality;
  • Measurement of information flow in multimodal neural datasets;
  • Relationships between artificial and natural neural systems.

Prof. Pulkit Grover
Prof. Christopher Quinn
Dr. Gabriel Schamberg
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Information flow
  • Neuroscience
  • Artificial neural networks
  • Information theory
  • Control and dynamical systems
  • Integrated information

Published Papers (2 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

17 pages, 1110 KiB  
Article
Redundant Information Neural Estimation
by Michael Kleinman, Alessandro Achille, Stefano Soatto and Jonathan C. Kao
Entropy 2021, 23(7), 922; https://doi.org/10.3390/e23070922 - 20 Jul 2021
Cited by 4 | Viewed by 3497
Abstract
We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, known as the “redundant information”. We show that existing definitions of the [...] Read more.
We introduce the Redundant Information Neural Estimator (RINE), a method that allows efficient estimation for the component of information about a target variable that is common to a set of sources, known as the “redundant information”. We show that existing definitions of the redundant information can be recast in terms of an optimization over a family of functions. In contrast to previous information decompositions, which can only be evaluated for discrete variables over small alphabets, we show that optimizing over functions enables the approximation of the redundant information for high-dimensional and continuous predictors. We demonstrate this on high-dimensional image classification and motor-neuroscience tasks. Full article
(This article belongs to the Special Issue Information Flow in Neural Systems)
►▼ Show Figures

Figure 1

21 pages, 2304 KiB  
Article
Heterogeneous Graphical Granger Causality by Minimum Message Length
by Kateřina Hlaváčková-Schindler and Claudia Plant
Entropy 2020, 22(12), 1400; https://doi.org/10.3390/e22121400 - 11 Dec 2020
Cited by 3 | Viewed by 4657
Abstract
The heterogeneous graphical Granger model (HGGM) for causal inference among processes with distributions from an exponential family is efficient in scenarios when the number of time observations is much greater than the number of time series, normally by several orders of magnitude. However, [...] Read more.
The heterogeneous graphical Granger model (HGGM) for causal inference among processes with distributions from an exponential family is efficient in scenarios when the number of time observations is much greater than the number of time series, normally by several orders of magnitude. However, in the case of “short” time series, the inference in HGGM often suffers from overestimation. To remedy this, we use the minimum message length principle (MML) to determinate the causal connections in the HGGM. The minimum message length as a Bayesian information-theoretic method for statistical model selection applies Occam’s razor in the following way: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct. Based on the dispersion coefficient of the target time series and on the initial maximum likelihood estimates of the regression coefficients, we propose a minimum message length criterion to select the subset of causally connected time series with each target time series and derive its form for various exponential distributions. We propose two algorithms—the genetic-type algorithm (HMMLGA) and exHMML to find the subset. We demonstrated the superiority of both algorithms in synthetic experiments with respect to the comparison methods Lingam, HGGM and statistical framework Granger causality (SFGC). In the real data experiments, we used the methods to discriminate between pregnancy and labor phase using electrohysterogram data of Islandic mothers from Physionet databasis. We further analysed the Austrian climatological time measurements and their temporal interactions in rain and sunny days scenarios. In both experiments, the results of HMMLGA had the most realistic interpretation with respect to the comparison methods. We provide our code in Matlab. To our best knowledge, this is the first work using the MML principle for causal inference in HGGM. Full article
(This article belongs to the Special Issue Information Flow in Neural Systems)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-2
Back to TopTop