Kybernetika 47 no. 4, 572-594, 2011

Bayesian estimation of mixtures with dynamic transitions and known component parameters

Ivan Nagy, Evgenia Suzdaleva and Miroslav Kárný

Abstract:

Probabilistic mixtures provide flexible "universal'' approximation of probability density functions. Their wide use is enabled by the availability of a range of efficient estimation algorithms. Among them, quasi-Bayesian estimation plays a prominent role as it runs "naturally'' in one-pass mode. This is important in on-line applications and/or extensive databases. It even copes with dynamic nature of components forming the mixture. However, the quasi-Bayesian estimation relies on mixing via constant component weights. Thus, mixtures with dynamic components and dynamic transitions between them are not supported. The present paper fills this gap. For the sake of simplicity and to give a better insight into the task, the paper considers mixtures with known components. A general case with unknown components will be presented soon.

Keywords:

approximation, mixture model, Bayesian estimation, clustering, classification

Classification:

93E12, 68T05

References:

  1. J. M. Bernardo: Expected information as expected utility. Ann. Statist. 7 (1979), 3, 686-690.   CrossRef
  2. J. B{ö}hm and M. Kárný: Transformation of user's knowledge into initial values for identification. In: {Preprints DYCOMANS Workshop Industrial Control and Management Methods: Theory and Practice} (M. Součková and J. B{ö}hm, eds.), ÚTIA AV ČR, Prague 1995, pp. 17-24.   CrossRef
  3. W. Chen and P. Jovanis: Method for identifying factors contributing to driver-injury severity in traffic crashes. Highway And Traffic Safety: Crash Data, Analysis Tools, And Statistical Methods 1717 (2000), 1-9.   CrossRef
  4. G. Chinnaswamy, E. Chirwa, S. Nammi, S. Nowpada, T. Chen and M. Mao: Benchmarking and accident characteristics of flat-fronted commercial vehicles with respect to pedestrian safety. Internat. J. Crashworthiness 12 (2007), 279-291.   CrossRef
  5. A. P. Dempster, N. Laird and D. Rubin: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B (Methodological) 39 (1977), 1, 1-38.   CrossRef
  6. J. D. Hamilton and R. Susmel: Autoregressive conditional heteroskedasticity and changes in regime. J. Econometrics 64 (1994), 307-333.   CrossRef
  7. Neural Networks: A Comprehensive Foundation. MacMillan, New York 1994.   CrossRef
  8. Jianyong Wang, Yuzhou Zhang, Lizhu Zhou, G. Karypis and Charu C. Aggarwal: Contour: An efficient algorithm for discovering discriminating subsequences. In: Data Mining and Knowledge Discovery, Springer 18 (2009), 1, pp. 1-29.   CrossRef
  9. M. Kárný: Tools for computer-aided design of adaptive controllers. IEE Control Theory Appl. 150 (2003), 6, 643.   CrossRef
  10. M. Kárný, J. Böhm, T. V. Guy, L. Jirsa, I. Nagy, P. Nedoma and L. Tesař: Optimized Bayesian Dynamic Advising: Theory and Algorithms. Springer, London 2005.   CrossRef
  11. M. Kárný, J. Kadlec and E. L. Sutanto: Quasi-Bayes estimation applied to normal mixture. in In: Preprints 3rd European IEEE Workshop on Computer-Intensive Methods in Control and Data Processing (J. Rojíček, M. Valečková, M. Kárný, and K. Warwick, eds.), ÚTIA AV ČR, Prague 1998, pp. 77-82.   CrossRef
  12. M. Kárný, I. Nagy and J. Novovičová: Mixed-data multi-modelling for fault detection and isolation. Internat. J. Adaptive Control Signal Process. 16 (2002), 1, 61-83.   CrossRef
  13. D. F. Kerridge: Inaccuracy and Inference. J. Royal Statist. Soc. Ser. B (Methodological) 23 (1961), 1, 184-194.   CrossRef
  14. R. Kulhavý: A Bayes-closed approximation of recursive non-linear estimation. Internat. J. Adaptive Control Signal Process. 4 (1990), 271-285.   CrossRef
  15. L. Ljung: System Identification: Theory for the User. Prentice-Hall, London 1987.   CrossRef
  16. A. Oppenheim and A. Wilsky: Signals and Systems. Englewood Clifts, Jersey 1983.   CrossRef
  17. M. Opper and D. Saad: Advanced Mean Field Methods: Theory and Practice. The MIT Press, Cambridge 2001.   CrossRef
  18. H. B. Qu and B. G. Hu: Variational learning for Generalized Associative Functional Networks in modeling dynamic process of plant growth. Ecological Informatics 4 (2009), 3, 163-176.   CrossRef
  19. W. A. Roberts: Convex Functions. Academic Press, New York 1973.   CrossRef
  20. J. Sander, M. Ester, H.-P. Kriegel and X. Xu: Density-based clustering in spatial databases: The algorithm gdbscan and its applications. In: Data Mining and Knowledge Discovery, Springer, 2 (1998), 2, pp. 169-194.   CrossRef
  21. D. Titterington, A. Smith and U. Makov: Statistical Analysis of Finite Mixtures. John Wiley, New York 1985.   CrossRef
  22. Xiaowei Xu, J. Jäger and H.-P. Kriegel: A fast parallel clustering algorithm for large spatial databases. In: Data Mining and Knowledge Discovery, Springer, 3 (1999), 3, pp. 263-290.   CrossRef
  23. T. Zhang, R. Ramakrishnan and M. Livny: Birch: A new data clustering algorithm and its applications. In: Data Mining and Knowledge Discovery, Springer, 1 (1997), 2, pp. 141-182.   CrossRef