An Estimate of Mutual Information that Permits Closed-Form Optimisation
AbstractWe introduce a new estimate of mutual information between a dataset and a target variable that can be maximised analytically and has broad applicability in the field of machine learning and statistical pattern recognition. This estimate has previously been employed implicitly as an approximation to quadratic mutual information. In this paper we will study the properties of these estimates of mutual information in more detail, and provide a derivation from a perspective of pairwise interactions. From this perspective, we will show a connection between our proposed estimate and Laplacian eigenmaps, which so far has not been shown to be related to mutual information. Compared with other popular measures of mutual information, which can only be maximised through an iterative process, ours can be maximised much more efficiently and reliably via closed-form eigendecomposition.
Scifeed alert for new publicationsNever miss any articles matching your research from any publisher
- Get alerts for new papers matching your research
- Find out the new papers from selected authors
- Updated daily for 49'000+ journals and 6000+ publishers
- Define your Scifeed now
Liu, R.; Gillies, D.F. An Estimate of Mutual Information that Permits Closed-Form Optimisation. Entropy 2013, 15, 1690-1704.
Liu R, Gillies DF. An Estimate of Mutual Information that Permits Closed-Form Optimisation. Entropy. 2013; 15(5):1690-1704.Chicago/Turabian Style
Liu, Raymond; Gillies, Duncan F. 2013. "An Estimate of Mutual Information that Permits Closed-Form Optimisation." Entropy 15, no. 5: 1690-1704.