Next Article in Journal
Interval Entropy and Informative Distance
Next Article in Special Issue
A Free Energy Principle for Biological Systems
Previous Article in Journal
Temporal Asymmetry, Entropic Irreversibility, and Finite-Time Thermodynamics: From Parmenides–Einstein Time-Reversal Symmetry to the Heraclitan Entropic Arrow of Time
Article Menu

Export Article

Open AccessArticle
Entropy 2012, 14(3), 456-479; doi:10.3390/e14030456

The Mathematical Structure of Information Bottleneck Methods

Department of Mathematical Sciences, Montana State University, Bozeman, MT 59717, USA
Center for Biofilm Engineering, Montana State University, Bozeman, MT 59717, USA
Department of Mathematics and Science Programs, Washington State University Vancouver, Vancouver, WA 98686, USA
Author to whom correspondence should be addressed.
Received: 2 December 2011 / Revised: 7 February 2012 / Accepted: 24 February 2012 / Published: 1 March 2012
(This article belongs to the Special Issue The Information Bottleneck Method)
View Full-Text   |   Download PDF [221 KB, uploaded 24 February 2015]   |  


Information Bottleneck-based methods use mutual information as a distortion function in order to extract relevant details about the structure of a complex system by compression. One of the approaches used to generate optimal compressed representations is by annealing a parameter. In this manuscript we present a common framework for the study of annealing in information distortion problems. We identify features that should be common to any annealing optimization problem. The main mathematical tools that we use come from the analysis of dynamical systems in the presence of symmetry (equivariant bifurcation theory). Through the compression problem, we make connections to the world of combinatorial optimization and pattern recognition. The two approaches use very different vocabularies and consider different problems to be “interesting”. We provide an initial link, through the Normalized Cut Problem, where the two disciplines can exchange tools and ideas. View Full-Text
Keywords: information distortion; spontaneous symmetry breaking; bifurcations; phase transition information distortion; spontaneous symmetry breaking; bifurcations; phase transition

Figure 1

This is an open access article distributed under the Creative Commons Attribution License (CC BY 3.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Gedeon, T.; Parker, A.E.; Dimitrov, A.G. The Mathematical Structure of Information Bottleneck Methods. Entropy 2012, 14, 456-479.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top