Kybernetika 48 no. 4, 600-636, 2012

Several applications of divergence criteria in continuous families

Michel Broniatowski and Igor Vajda


This paper deals with four types of point estimators based on minimization of information-theoretic divergences between hypothetical and empirical distributions. These were introduced \begin{enumerate} \item[(i)] by Liese and Vajda \cite{9} and independently Broniatowski and Keziou \cite{3}, called here \textsl{power superdivergence estimators, } \item[(ii)] by Broniatowski and Keziou \cite{4} , called here \textsl{power subdivergence estimators, } \item[(iii)] by Basu et al. \cite{2}, called here \textsl{power pseudodistance estimators, }and \item[(iv)] by Vajda \cite{18} called here \textsl{Rényi pseudodistance estimators.} \end{enumerate} These various criterions have in common to eliminate all need for grouping or smoothing in statistical inference. The paper studies and compares general properties of these estimators such as Fisher consistency and influence curves, and illustrates these properties by detailed analysis of the applications to the estimation of normal location and scale.


robustness, divergence, parametric estimation


62B10, 62F10, 62F35


  1. D. F. Andrews, P. J. Bickel, F. R. Hampel, P. J. Huber, W. H. Rogers and J. W. Tukey: Robust Estimates of Location. Princeton University Press, Princeton N. J. 1972.   CrossRef
  2. A. Basu, I. R. Harris, N. L. Hjort and M. C. Jones: Robust and efficient estimation by minimizing a density power divergence. Biometrika 85 (1998), 3, 549-559.   CrossRef
  3. M. Broniatowski and A. Keziou: Minimization of $\phi$-divergences on sets of signed measures. Studia Sci. Math. Hungar. 43 (2006), 403-442.   CrossRef
  4. M. Broniatowski and A. Keziou: Parametric estimation and tests through divergences and the duality technique. J. Multivariate Anal. 100 (2009), 1, 16-31.   CrossRef
  5. M. Broniatowski, A. Toma and I. Vajda: Decomposable pseudodistances and applications in statistical estimation. J. Statist. Plann. Inference. 142 (2012), 9, 2574-2585   CrossRef
  6. M. Broniatowski and I. Vajda: Several applications of divergence criteria in continuous families. arXiv:0911.0937v1, 2009.   CrossRef
  7. F. R. Hampel, E. M. Ronchetti, P. J. Rousseuw and W. A. Stahel: Robust Statistics: The approach Based on Influence Functions. Willey, New York 1986.   CrossRef
  8. F. Liese and I. Vajda: Convex Statistical Distances. Teubner, Leipzig 1987.   CrossRef
  9. F. Liese and I. Vajda: On divergences and informations in statistics and information theory. IEEE Trans. Inform. Theory 52 (2006), 10, 4394-4412.   CrossRef
  10. C. Miescke and F. Liese: Statistical Decision Theory. Springer, Berlin 2008.   CrossRef
  11. M. R. C. Read and N. A. C. Cressie: Goodness-of-Fit Statistics for Discrete Multivariate Data. Springer, Berlin 1988.   CrossRef
  12. A. Rényi: On measures of entropy and information. In: Proc. 4th Berkeley Symp. on Probability and Statistics, Vol. 1, University of California Press, Berkeley 1961, pp. 547-561.   CrossRef
  13. A. Toma and M. Broniatowski: Minimum divergence estimators and tests: Robustness results. J. Multivariate Anal. 102 (2011), 1, 20-36.   CrossRef
  14. I. Vajda: Minimum divergence principle in statistical estimation. Statist. Decisions (1984), Suppl. Issue No. 1, 239-261.   CrossRef
  15. I. Vajda: Efficiency and robustness control via distorted maximum likelihood estimation. Kybernetika 22 (1986), 47-67.   CrossRef
  16. I. Vajda: Comparison of asymptotic variances for several estimators of location. Probl. Control Inform. Theory 18 (1989), 2, 79-89.   CrossRef
  17. I. Vajda: Estimators asymptotically minimax in wide sense. Biometr. J. 31 (1989), 7, 803-810.   CrossRef
  18. I. Vajda: Modifications od Divergence Criteria for Applications in Continuous Families. Research Report No. 2230, Institute of Information Theory and Automation, Prague 2008.   CrossRef
  19. A. W. van der Vaart: Asymptotic Statistics. Cambridge University Press, Cambridge 1998.   CrossRef
  20. A. W. van der Vaart and J. A. Wellner: Weak Convergence and Empirical Processes. Springer, Berlin 1996.   CrossRef