A Measure of Overall Statistical Dependence Based on the
Entropy Concept
Jaakko Astola and Ilkka Virtanen
Abstract
The main use of Shannonıs entropy in statistics has been
in measuring the dispersion of one-dimensional categorized data.
However, entropy can also be defined for a two- or multi-
dimensional distribution given as a contingency table. This
generalized entropy, called coentropy, forms a basis for a
measure of overall dependence between the variables in the table.
By reducing the lower order entropies from the coentropy and
using an appropriate scaling such a measure of dependence, that
fulfills the criteria for a well-defined correlation coefficient, can be
constructed. This entropy correlation coefficient is
introduced and then analyzed in this paper.
(Proceedings of The University of Vaasa, Research
Papers No. 91 (1983), 34 p.)