Kybernetika 56 no. 6, 1111-1132, 2020

Bounds on the information divergence for hypergeometric distributions

Peter Harremoës and František MatúšDOI: 10.14736/kyb-2020-6-1111

Abstract:

The hypergeometric distributions have many important applications, but they have not had sufficient attention in information theory. Hypergeometric distributions can be approximated by binomial distributions or Poisson distributions. In this paper we present upper and lower bounds on information divergence. These bounds are important for statistical testing and for a better understanding of the notion of exchangeability.

Keywords:

information divergence, inequalities, hypergeometric distribution, binomial distribution

Classification:

62E17, 94A17

paper.pdf

References:

  1. A. D. Barbour, L. Holst, L. and S. Janson: Poisson Approximation. Oxford Studies in Probability 2, Clarendon Press, Oxford 1992.   DOI:10.1002/bimj.4710350414
  2. T. M. Cover and J. A. Thomas: Elements of Information Theory. Wiley Series in Telecommunications. 1991.   DOI:10.1002/047174882x
  3. I. Csiszár and P. Shields: Information Theory and Statistics: A Tutorial. Foundations and Trends in Communications and Information Theory, Now Publishers Inc., (2004) 4, 417-528.   DOI:10.1561/0100000004
  4. P. Diaconis and D. Friedman: A dozen de Finetti-style results in search of a theory Ann. Inst. Henri Poincaré 23 (1987), 2, 397-423.   CrossRef
  5. P. Harremoës: Mutual information of contingency tables and related inequalities. In: 2014 IEEE International Symposium on Information Theory, IEEE 2014, pp. 2474-2478.   DOI:10.1109/isit.2014.6875279
  6. P. Harremoës, O. Johnson and I. Kontoyiannis: Thinning and information projections. arXiv:1601.04255, 2016.   CrossRef
  7. P. Harremoës and P. Ruzankin: Rate of Convergence to Poisson Law in Terms of Information Divergence. IEEE Trans. Inform Theory 50 (2004), 9, 2145-2149.   DOI:10.1109/tit.2004.833364
  8. F. Matúš: Urns and entropies revisited. In: 2017 IEEE International Symposium on Information Theory (ISIT) 2017, pp. 1451-1454.   DOI:https://doi.org/10.1109/isit.2017.8006769
  9. A. J. Stam: Distance between sampling with and without replacement. Statistica Neerlandica 32 (1978), 2, 81-91.   DOI:10.1111/j.1467-9574.1978.tb01387.x