Extending Qualitative Probabilistic Network with Mutual Information Weights
Affiliation(s)
1 Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming, China. 2 Yunnan Computer Technology Application Key Lab, Kunming University of Science and Technology, Kunming, China.
1 Department of Computer Science and Engineering, School of Information Science and Engineering, Yunnan University, Kunming, China. 2 Yunnan Computer Technology Application Key Lab, Kunming University of Science and Technology, Kunming, China.
ABSTRACT
Bayesian network (BN) is a well-accepted framework for representing and inferring uncertain knowledge. As the qualitative abstraction of BN, qualitative probabilistic network (QPN) is introduced for probabilistic inferences in a qualitative way. With much higher efficiency of inferences, QPNs are more suitable for real-time applications than BNs. However, the high abstraction level brings some inference conflicts and tends to pose a major obstacle to their applications. In order to eliminate the inference conflicts of QPN, in this paper, we begin by extending the QPN by adding a mutual-information-based weight (MI weight) to each qualitative influence in the QPN. The extended QPN is called MI-QPN. After obtaining the MI weights from the corresponding BN, we discuss the symmetry, transitivity and composition properties of the qualitative influences. Then we extend the general inference algorithm to implement the conflict-free inferences of MI-QPN. The feasibility of our method is verified by the results of the experiment.
Bayesian network (BN) is a well-accepted framework for representing and inferring uncertain knowledge. As the qualitative abstraction of BN, qualitative probabilistic network (QPN) is introduced for probabilistic inferences in a qualitative way. With much higher efficiency of inferences, QPNs are more suitable for real-time applications than BNs. However, the high abstraction level brings some inference conflicts and tends to pose a major obstacle to their applications. In order to eliminate the inference conflicts of QPN, in this paper, we begin by extending the QPN by adding a mutual-information-based weight (MI weight) to each qualitative influence in the QPN. The extended QPN is called MI-QPN. After obtaining the MI weights from the corresponding BN, we discuss the symmetry, transitivity and composition properties of the qualitative influences. Then we extend the general inference algorithm to implement the conflict-free inferences of MI-QPN. The feasibility of our method is verified by the results of the experiment.
KEYWORDS
Qualitative Probabilistic Network (QPN), Inference Conflict, Mutual Information, Influence Weight, Superposition
Qualitative Probabilistic Network (QPN), Inference Conflict, Mutual Information, Influence Weight, Superposition
Cite this paper
Yue, K. , Wang, F. , Wei, M. and Liu, W. (2015) Extending Qualitative Probabilistic Network with Mutual Information Weights. International Journal of Intelligence Science, 5, 133-144. doi: 10.4236/ijis.2015.53012.
Yue, K. , Wang, F. , Wei, M. and Liu, W. (2015) Extending Qualitative Probabilistic Network with Mutual Information Weights. International Journal of Intelligence Science, 5, 133-144. doi: 10.4236/ijis.2015.53012.
References
[1] Liu, W.Y., Li, W.H. and Yue, K. (2007) Intelligent Data Analysis. Science Press, Beijing. [2] Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference. Morgan Kaufmann, San Mateo. [3] Li, W.H., Liu, W.Y. and Yue, K. (2008) Recovering the Global Structure from Multiple Local Bayesian Networks. International Journal on Artificial Intelligence Tools, 17, 1067-1088.
http://dx.doi.org/10.1142/S0218213008004308 [4] Cooper, G.F. (1990) The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks. Artificial Intelligence, 42, 393-405.
http://dx.doi.org/10.1016/0004-3702(90)90060-D [5] Dagum, P. and Luby, M. (1993) Approximating Probabilistic Inference in Bayesian Belief Networks Is NP-Hard. Artificial Intelligence, 60, 141-153.
http://dx.doi.org/10.1016/0004-3702(93)90036-B [6] Wellman, M.P. (1990) Fundamental Concepts of Qualitative Probabilistic Networks. Artificial Intelligence, 44, 257-303.
http://dx.doi.org/10.1016/0004-3702(90)90026-V [7] Bolt, J.H., Renooij, S. and Van der Gaag, L.C. (2003) Upgrading Ambiguous Signs in QPNs. Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence, Acapulco, 7-10 August 2003, 73-80. [8] Druzdzel, M.J. and Henrion, M. (1993) Efficient Reasoning in Qualitative Probabilistic Networks. Proceedings of the 11th National Conference on Artificial Intelligence, Washington DC, 11-15 July 1993, 548-553. [9] Lv, Y.L. and Liao, S.Z. (2011) Ambiguity Reduction Based on Qualitative Mutual Information in Qualitative Probabilistic Networks. Pattern Recognition and Artificial Intelligence, 24, 123-129. [10] Parsons, S. (1995) Refining Reasoning in Qualitative Probabilistic Networks. Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, Montreal, 18-20 August 1995, 427-434. [11] Renooij, S., Parsons, S. and Pardieck, P. (2003) Using Kappas as Indicators of Strength in Qualitative Probabilistic Networks. Proceedings of European Conferences on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, Aalborg, 2-5 July 2003, 87-99. [12] Renooij, S. and Van der Gaag, L.C. (1999) Enhancing QPNs for Trade-Off Resolution. Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, Stockholm, 30 July-1 August 1999, 559-566. [13] Renooij, S., Van der Gaag, L.C. and Parsons, S. (2002) Context-Specific Sign-Propagation in Qualitative Probabilistic Networks. Artificial Intelligence, 140, 207-230.
http://dx.doi.org/10.1016/S0004-3702(02)00247-3 [14] Renooij, S., Van der Gaag, L.C., Parsons, S. and Green, S. (2000) Pivotal Pruning of Trade-Off in QPNs. Proceedings of the 16th Conference in Uncertainty in Artificial Intelligence, Stanford, 30 June-3 July 2000, 515-522. [15] Renooij, S. and Van der Gaag, L.C. (2002) From Qualitative to Quantitative Probabilistic Network. Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, Edmonton, 1-4 August 2002, 422-429. [16] Yue, K., Yao, Y., Li, J. and Liu, W.Y. (2010) Qualitative Probabilistic Networks with Reduced Ambiguities. Applied Intelligence, 33, 159-178.
http://dx.doi.org/10.1007/s10489-008-0156-5 [17] Cover, T.M. and Thomas, J.A. (1993) Elements of Information Theory. John Wiley & Sons, Inc., Hoboken. [18] Shannon, C.E. and Weaver, W. (1949) The Mathematical Theory of Communication. University of Illinois Press, Champaign. [19] Chen, X.W., Anantha, G. and Lin, X.T. (2008) Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm. IEEE Transactions on Knowledge and Data Engineering, 20, 628-640.
http://dx.doi.org/10.1109/TKDE.2007.190732 [20] De Campos, L.M. (2006) A Scoring Function for Learning Bayesian Networks Based on Mutual Information and Conditional Independence Tests. Journal of Machine Learning Research, 7, 2149-2187. [21] Nicholson, A.E. and Jitnah, N. (1998) Using Mutual Information to Determine Relevance in Bayesian Networks. Proceedings of the 5th Pacific Rim International Conference on Artificial Intelligence, Singapore, 22-27 November 1998, 399-410. [22] Ibrahim, Z.M., Ngom, A. and Tawk, A.Y. (2011) Using Qualitative Probability in Reverse-Engineering Gene Regulatory Networks. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 8, 326-334.
http://dx.doi.org/10.1109/TCBB.2010.98 [23] Liu, W.Y., Yue, K., Liu, S.X. and Sun, Y.B. (2008) Qualitative-Probabilistic-Network-Based Modeling of Temporal Causalities and Its Application to Feedback Loop Identification. Information Sciences, 178, 1803-1824.
http://dx.doi.org/10.1016/j.ins.2007.11.021 [24] Yue, K., Qian, W.H., Fu, X.D., Li, J. and Liu, W.Y. (2014) Qualitative-Probabilistic-Network-Based Fusion of Time-Series Uncertain Knowledge. Soft Computing, Published Online.
http://dx.doi.org/10.1007/s00500-014-1381-y. [25] De Campos, C.P. and Cozman, F.G. (2013) Complexity of Inferences in Polytree-Shaped Semi-Qualitative Probabilistic Networks. Proceedings of the 27th AAAI Conference on Artificial Intelligence, Bellevue, 14-18 July 2013, 217-223. [26] Yue, K., Liu, W. and Yue, M. (2011) Quantifying Influences in the Qualitative Probabilistic Network with Interval Probability Parameters. Applied Soft Computing, 11, 1135-1143.
http://dx.doi.org/10.1016/j.asoc.2010.02.013 [27] Shafer, G. (1986) The Combination of Evidence. International Journal of Intelligent Systems, 1, 155-179.
http://dx.doi.org/10.1002/int.4550010302 [28] Norsys Software Corp. (2007) Netica 3.17 Bayesian Network Software from Norsys.
http://www.norsys.com
[1] Liu, W.Y., Li, W.H. and Yue, K. (2007) Intelligent Data Analysis. Science Press, Beijing. [2] Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference. Morgan Kaufmann, San Mateo. [3] Li, W.H., Liu, W.Y. and Yue, K. (2008) Recovering the Global Structure from Multiple Local Bayesian Networks. International Journal on Artificial Intelligence Tools, 17, 1067-1088.
http://dx.doi.org/10.1142/S0218213008004308 [4] Cooper, G.F. (1990) The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks. Artificial Intelligence, 42, 393-405.
http://dx.doi.org/10.1016/0004-3702(90)90060-D [5] Dagum, P. and Luby, M. (1993) Approximating Probabilistic Inference in Bayesian Belief Networks Is NP-Hard. Artificial Intelligence, 60, 141-153.
http://dx.doi.org/10.1016/0004-3702(93)90036-B [6] Wellman, M.P. (1990) Fundamental Concepts of Qualitative Probabilistic Networks. Artificial Intelligence, 44, 257-303.
http://dx.doi.org/10.1016/0004-3702(90)90026-V [7] Bolt, J.H., Renooij, S. and Van der Gaag, L.C. (2003) Upgrading Ambiguous Signs in QPNs. Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence, Acapulco, 7-10 August 2003, 73-80. [8] Druzdzel, M.J. and Henrion, M. (1993) Efficient Reasoning in Qualitative Probabilistic Networks. Proceedings of the 11th National Conference on Artificial Intelligence, Washington DC, 11-15 July 1993, 548-553. [9] Lv, Y.L. and Liao, S.Z. (2011) Ambiguity Reduction Based on Qualitative Mutual Information in Qualitative Probabilistic Networks. Pattern Recognition and Artificial Intelligence, 24, 123-129. [10] Parsons, S. (1995) Refining Reasoning in Qualitative Probabilistic Networks. Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, Montreal, 18-20 August 1995, 427-434. [11] Renooij, S., Parsons, S. and Pardieck, P. (2003) Using Kappas as Indicators of Strength in Qualitative Probabilistic Networks. Proceedings of European Conferences on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, Aalborg, 2-5 July 2003, 87-99. [12] Renooij, S. and Van der Gaag, L.C. (1999) Enhancing QPNs for Trade-Off Resolution. Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, Stockholm, 30 July-1 August 1999, 559-566. [13] Renooij, S., Van der Gaag, L.C. and Parsons, S. (2002) Context-Specific Sign-Propagation in Qualitative Probabilistic Networks. Artificial Intelligence, 140, 207-230.
http://dx.doi.org/10.1016/S0004-3702(02)00247-3 [14] Renooij, S., Van der Gaag, L.C., Parsons, S. and Green, S. (2000) Pivotal Pruning of Trade-Off in QPNs. Proceedings of the 16th Conference in Uncertainty in Artificial Intelligence, Stanford, 30 June-3 July 2000, 515-522. [15] Renooij, S. and Van der Gaag, L.C. (2002) From Qualitative to Quantitative Probabilistic Network. Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, Edmonton, 1-4 August 2002, 422-429. [16] Yue, K., Yao, Y., Li, J. and Liu, W.Y. (2010) Qualitative Probabilistic Networks with Reduced Ambiguities. Applied Intelligence, 33, 159-178.
http://dx.doi.org/10.1007/s10489-008-0156-5 [17] Cover, T.M. and Thomas, J.A. (1993) Elements of Information Theory. John Wiley & Sons, Inc., Hoboken. [18] Shannon, C.E. and Weaver, W. (1949) The Mathematical Theory of Communication. University of Illinois Press, Champaign. [19] Chen, X.W., Anantha, G. and Lin, X.T. (2008) Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm. IEEE Transactions on Knowledge and Data Engineering, 20, 628-640.
http://dx.doi.org/10.1109/TKDE.2007.190732 [20] De Campos, L.M. (2006) A Scoring Function for Learning Bayesian Networks Based on Mutual Information and Conditional Independence Tests. Journal of Machine Learning Research, 7, 2149-2187. [21] Nicholson, A.E. and Jitnah, N. (1998) Using Mutual Information to Determine Relevance in Bayesian Networks. Proceedings of the 5th Pacific Rim International Conference on Artificial Intelligence, Singapore, 22-27 November 1998, 399-410. [22] Ibrahim, Z.M., Ngom, A. and Tawk, A.Y. (2011) Using Qualitative Probability in Reverse-Engineering Gene Regulatory Networks. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 8, 326-334.
http://dx.doi.org/10.1109/TCBB.2010.98 [23] Liu, W.Y., Yue, K., Liu, S.X. and Sun, Y.B. (2008) Qualitative-Probabilistic-Network-Based Modeling of Temporal Causalities and Its Application to Feedback Loop Identification. Information Sciences, 178, 1803-1824.
http://dx.doi.org/10.1016/j.ins.2007.11.021 [24] Yue, K., Qian, W.H., Fu, X.D., Li, J. and Liu, W.Y. (2014) Qualitative-Probabilistic-Network-Based Fusion of Time-Series Uncertain Knowledge. Soft Computing, Published Online.
http://dx.doi.org/10.1007/s00500-014-1381-y. [25] De Campos, C.P. and Cozman, F.G. (2013) Complexity of Inferences in Polytree-Shaped Semi-Qualitative Probabilistic Networks. Proceedings of the 27th AAAI Conference on Artificial Intelligence, Bellevue, 14-18 July 2013, 217-223. [26] Yue, K., Liu, W. and Yue, M. (2011) Quantifying Influences in the Qualitative Probabilistic Network with Interval Probability Parameters. Applied Soft Computing, 11, 1135-1143.
http://dx.doi.org/10.1016/j.asoc.2010.02.013 [27] Shafer, G. (1986) The Combination of Evidence. International Journal of Intelligent Systems, 1, 155-179.
http://dx.doi.org/10.1002/int.4550010302 [28] Norsys Software Corp. (2007) Netica 3.17 Bayesian Network Software from Norsys.
http://www.norsys.com