Kybernetika 61 no. 4, 481-491, 2025

Metrization of powers of the Jensen-Shannon divergence

Kazuki OkamuraDOI: 10.14736/kyb-2025-4-0481

Abstract:

Metrization of statistical divergences is valuable in both theoretical and practical aspects. One approach to obtaining metrics associated with divergences is to consider their fractional powers. Motivated by this idea, Os{\'an}, Bussandri, and Lamberti (2018) studied the metrization of fractional powers of the Jensen--Shannon divergence between multinomial distributions and posed an open problem. In this short note, we provide an affirmative answer to their conjecture. Moreover, our method is also applicable to fractional powers of $f$-divergences between Cauchy distributions.

Keywords:

multinomial distribution, Jensen-Shannon divergence, metric, Cauchy distribution

Classification:

94A17, 60A10, 60E05, 28A33

paper.pdf

References:

  1. S. Acharyya, A. Banerjee and D. Boley: Bregman divergences and triangle inequality. In: Proc. 2013 SIAM International Conference on Data Mining, SIAM 2013, pp. 476-484. SIAM.   CrossRef
  2. K. Chrz{\c a}szcz, J. Jachymski and F. Turoboś: On characterizations and topology of regular semimetric spaces. Publ. Math. Debr. 93 (2018), 87-105.   DOI:10.5486/PMD.2018.8049
  3. M. de Berg, M. van Kreveld, M. Overmars and O. Schwarzkopf: Computational Geometry. {Algorithms} and Applications. (Second rev. edition.) Springer, Berlin 2000.   CrossRef
  4. W. Ehm, M. G. Genton and T. Gneiting: Stationary covariances associated with exponentially convex functions. Bernoulli 9 (2004), 607-615.   DOI:10.3150/bj/1066223271
  5. C. Elkan: Using the triangle inequality to accelerate $k$-means. In: Proc. 20th International Conference on Machine Learning (ICML-03), 2003, pp. 147-153.   CrossRef
  6. D. M. Endres and J. E. Schindelin: A new metric for probability distributions. IEEE Trans. Inform. Theory 49 (2003), 1858-1860.   DOI:10.1109/TIT.2003.813506
  7. B. Fuglede and F. Topsøe: Jensen-Shannon divergence and Hilbert space embedding. In: Proc. International Symposium on Information Theory, 2004. ISIT, IEEE 2004, p. 31.   CrossRef
  8. I. Grosse, P. Bernaola-Galván, P. Carpena, R. Román-Roldán, J. Oliver and H. E. Stanley: Analysis of symbolic sequences using the Jensen-Shannon divergence. Phys. Rev. E (3), 65 (2002).   DOI:10.1103/physreve.65.041905
  9. P. Kafka, F. Österreicher and I. Vincze: On powers of $f$-divergences defining a distance. Stud. Sci. Math. Hung. 26 (1991), 415-422.   DOI:10.3109/10826089109058894
  10. G. A. Kalugin and D. J. Jeffrey: Unimodal sequences show that Lambert {$W$} is Bernstein. C. R. Math. Acad. Sci. Soc. R. Can. 33 (2011), 50-56.   CrossRef
  11. J. Lin: Divergence measures based on the Shannon entropy. IEEE Trans. Inform. Theory 37 (1991), 145-151.   DOI:10.1109/18.61115
  12. P. McCullagh: Möbius transformation and {C}auchy parameter estimation. Ann. Statist. 24 (1996), 787-808.   CrossRef
  13. F. Nielsen: On the Jensen-Shannon symmetrization of distances relying on abstract means. Entropy 21 (2019), 485.   DOI:10.3390/e21050485
  14. F. Nielsen and K. Okamura: On $f$-divergences between {Cauchy} distributions. IEEE Trans. Inform. Theory 69 (2023), 3150-3171.   DOI:10.1109/TIT.2022.3231645
  15. T. M. Osán, D. G. Bussandri and P. W. Lamberti: Monoparametric family of metrics derived from classical Jensen-Shannon divergence. Physica A, 495 (2018), 336-344.   DOI:10.1016/j.physa.2017.12.073
  16. T. M. Osán, D. G. Bussandri and P. W. Lamberti: Quantum metrics based upon classical Jensen-Shannon divergence. Physica A 594 (2022).   DOI:10.1016/j.physa.2022.127001
  17. F. Österreicher and I. Vajda: A new class of metric divergences on probability spaces and its applicability in statistics. Ann. Inst. Stat. Math. 55 (2003), 639-653.   DOI:10.1007/BF02517812
  18. S. T. Rachev, L. B. Klebanov, S. V. Stoyanov and F. J. Fabozzi: The Methods of Distances in the Theory of Probability and Statistics. Springer, New York 2013.   CrossRef
  19. R. L. Schilling, R. Song and Z. Vondraček: Bernstein functions, volume 37 of De Gruyter Studies in Mathematics. Walter de Gruyter Co., Berlin 2010.   CrossRef
  20. I. J. Schoenberg: Metric spaces and positive definite functions. Trans. Amer. Math. Soc. 44 (1938), 522-536.   DOI:10.1090/S0002-9947-1938-1501980-0
  21. I. Vajda: On metric divergences of probability measures. Kybernetika 45 (2009), 885-900.   DOI:10.1145/1932682.1869533
  22. S. Verdú: The Cauchy distribution in information theory. Entropy 25 (2023), 346.   DOI:10.3390/e25020346
  23. P. N. Yianilos: Data structures and algorithms for nearest neighbor. In: Proce. Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM 66 (1993), p. 311.   CrossRef
  24. V. M. Zolotarev: One-Dimensional Stable Distributions. Americal Mathematical Society, 1986.   CrossRef