Kybernetika 51 no. 3, 508-524, 2015

Generalizations of the noisy-or model

Jiří VomlelDOI: 10.14736/kyb-2015-3-0508

Abstract:

In this paper, we generalize the noisy-or model. The generalizations are three-fold. First, we allow parents to be multivalued ordinal variables. Second, parents can have both positive and negative influences on their common child. Third, we describe how the suggested generalization can be extended to multivalued child variables. The major advantage of our generalizations is that they require only one parameter per parent. We suggest a model learning method and report results of experiments on the Reuters text classification data. The generalized noisy-or models achieve equal or better performance than the standard noisy-or. An important property of the noisy-or model and of its generalizations suggested in this paper is that it allows more efficient exact inference than logistic regression models do.

Keywords:

Bayesian networks, classification, generalized linear models, noisy-or model

Classification:

68T37, 68T30

References:

  1. R. G. Almond, R. J. Mislevy, L. Steinberg, D. Yan and D. Williamson: Bayesian Networks in Educational Assessment. Statistics for Social and Behavioral Sciences. Springer, New York 2015.   DOI:10.1007/978-1-4939-2125-6
  2. Ch. Apt{é}, F. Damerau and S. M. Weiss: Automated learning of decision rules for text categorization. ACM Trans. Inform. Syst. 12 (1994), 3, 233-251.   DOI:10.1145/183422.183423
  3. R. H. Byrd, P. Lu, J. Nocedal and C. Zhu: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16 (1995), 1190-1208.   DOI:10.1137/0916069
  4. F. J. Díez: Parameter adjustment in Bayes networks. The generalized noisy OR gate. In: Proc. Ninth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann 1993, pp. 99-105.   DOI:10.1016/b978-1-4832-1451-1.50016-0
  5. F. J. Díez and M. J. Druzdzel: Canonical Probabilistic Models for Knowledge Engineering. Technical Report CISIAD-06-01, UNED, Madrid 2006.   CrossRef
  6. F. J. Díez and S. F. Galán: An efficient factorization for the noisy MAX. Int. J. Intell. Syst. 18 (2003), 165-177.   DOI:10.1002/int.10080
  7. D. Heckerman and J. Breese: A new look at causal independence. In: Proc. Tenth Conference on Uncertainty in Artificial Intelligence, Seattle, Morgan Kaufmann 1994, pp. 286-292.   DOI:10.1016/b978-1-55860-332-5.50041-9
  8. M. Henrion: Practical issues in constructing a Bayes' Belief Network. In: Proc. Third Conference Annual Conference on Uncertainty in Artificial Intelligence, AUAI Press 1987, pp. 132-139.   CrossRef
  9. F. V. Jensen and T. D. Nielsen: Bayesian Networks and Decision Graphs. Second edition. Springer, 2007.   DOI:10.1007/978-0-387-68282-2
  10. P. McCullagh: Regression models for ordinal data. J. Roy. Statist. Soc. Series B (Methodological) 42 (1980), 109-142.   CrossRef
  11. P. McCullagh and J. A. Nelder: Generalized Linear Models. Chapman and Hall, London 1989.   DOI:10.1007/978-1-4899-3242-6
  12. R. A. Miller, F. E. Fasarie and J. D. Myers: Quick medical reference (QMR) for diagnostic assistance. Medical Comput. 3 (1986), 34-48.   CrossRef
  13. R. M. Neal: Connectionist learning of belief networks. Artif. Intell. 56 (1992), 1, 71-113.   DOI:10.1016/0004-3702(92)90065-6
  14. J. Pearl: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufman, San Mateo 1988.   CrossRef
  15. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna 2008.   CrossRef
  16. F. Rijmen: Bayesian networks with a logistic regression model for the conditional probabilities. Int. J. Approx. Reas. 48 (2008), 2, 659-666.   CrossRef
  17. F. Samejima: Estimation of Latent Ability Using a Response Pattern of Raded Scores (Psychometric Monograph No. 17). Psychometric Society, Richmond 1969.   CrossRef
  18. L. K. Saul, T. Jaakkola and M. I. Jordan: Mean field theory for sigmoid belief networks. J. Artif. Intell. Res. 4 (1996), 61-76.   CrossRef
  19. P. Savický and J. Vomlel: Exploiting tensor rank-one decomposition in probabilistic inference. Kybernetika 43 (2007), 5, 747-764.   CrossRef
  20. S. Srinivas: A generalization of the noisy-or model. In: Proc. Ninth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann 1993, pp. 208-215.   DOI:10.1016/b978-1-4832-1451-1.50030-5
  21. J. Vomlel: Noisy-or classifier. Int. J. Intell. Syst. 21 (2006), 381-398.   DOI:10.1002/int.20141
  22. J. Vomlel: A generalization of the noisy-or model to multivalued parent variables. In: Proc. 16th Czech-Japan Seminar on Data Analysis and Decision Making under Uncertainty 2013, pp. 19-27.   CrossRef
  23. J. Vomlel and P. Tichavský: On tensor rank of conditional probability tables in Bayesian networks. A preprint arXiv:1409.6287, 2014.   CrossRef
  24. A. Zagorecki and M. J. Druzdzel: Knowledge engineering for Bayesian networks: How common are noisy-MAX distributions in practice? IEEE Trans. Systems, Man, and Cybernetics: Systems 43 (2013) 186-195.   DOI:10.1109/tsmca.2012.2189880