Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front Comput Sci    2013, Vol. 7 Issue (6) : 852-863    https://doi.org/10.1007/s11704-013-3007-4
RESEARCH ARTICLE
Hybrid Bayesian estimation tree learning with discrete and fuzzy labels
Zengchang QIN1(), Tao WAN2,3()
1. Intelligent Computing and Machine Learning Lab, School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China; 2. School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China; 3. Department of Biomedical Engineering, Case Western Reserve University, Cleveland OH 44106, USA
 Download: PDF(757 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Classical decision tree model is one of the classical machine learning models for its simplicity and effectiveness in applications. However, compared to the DT model, probability estimation trees (PETs) give a better estimation on class probability. In order to get a good probability estimation, we usually need large trees which are not desirable with respect to model transparency. Linguistic decision tree (LDT) is a PET model based on label semantics. Fuzzy labels are used for building the tree and each branch is associated with a probability distribution over classes. If there is no overlap between neighboring fuzzy labels, these fuzzy labels then become discrete labels and a LDT with discrete labels becomes a special case of the PET model. In this paper, two hybrid models by combining the naive Bayes classifier and PETs are proposed in order to build a model with good performance without losing too much transparency. The first model uses naive Bayes estimation given a PET, and the second model uses a set of small-sized PETs as estimators by assuming the independence between these trees. Empirical studies on discrete and fuzzy labels show that the first model outperforms the PET model at shallow depth, and the second model is equivalent to the naive Bayes and PET.

Keywords fuzzy labels      label semantics      random set      probability estimation tree      mass assignment      linguistic decision tree      naive Bayes     
Corresponding Author(s): QIN Zengchang,Email:zcqin@buaa.edu.cn; WAN Tao,Email:tao.wan.wan@gmail.com   
Issue Date: 01 December 2013
 Cite this article:   
Zengchang QIN,Tao WAN. Hybrid Bayesian estimation tree learning with discrete and fuzzy labels[J]. Front Comput Sci, 2013, 7(6): 852-863.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-013-3007-4
https://academic.hep.com.cn/fcs/EN/Y2013/V7/I6/852
1 Quinlan J R. Induction of decision trees.Machine Learning , 1986, 1(1): 81-106
doi: 10.1007/BF00116251
2 Olaru C, Wehenkel L. A complete fuzzy decision tree technique. Fuzzy Sets and Systems , 2003, 138(2): 221-254
doi: 10.1016/S0165-0114(03)00089-7
3 Quinlan J R. C4. 5: programs for machine learning. Morgan Kaufmann , 1993
4 Baldwin J, Lawry J, Martin T. Mass assignment fuzzy ID3 with applications. In: Proceedings of the Unicom Workshop on Fuzzy Logic: Applications and Future Directions . 1997, 278-294
5 Janikow C Z. Fuzzy decision trees: issues and methods. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics , 1998, 28(1): 1-14
doi: 10.1109/3477.658573
6 Huang Z, Gedeon T D, Nikravesh M. Pattern trees induction: a new machine learning method. IEEE Transactions on Fuzzy Systems , 2008, 16(4): 958-970
doi: 10.1109/TFUZZ.2008.924348
7 Qin B, Xia Y, Li F. Dtu: a decision tree for uncertain data. Advances in Knowledge Discovery and Data Mining , 2009: 4-15
8 Provost F, Domingos P. Tree induction for probability-based ranking. Machine Learning , 2003, 52(3): 199-215
doi: 10.1023/A:1024099825458
9 Qin Z, Lawry J. Decision tree learning with fuzzy labels. Information Sciences , 2005, 172(1): 91-129
doi: 10.1016/j.ins.2004.12.005
10 Qin Z, Lawry J. Prediction trees using linguistic modelling. In: Proceedings of World Congress of International Fuzzy Systems Association (IFSA-05) , 2005
11 Qin Z, Lawry J. Prediction and query evaluation using linguistic decision trees. Applied Soft Computing , 2011, 11(5): 3916-3928
doi: 10.1016/j.asoc.2011.02.010
12 Lawry J. A framework for linguistic modelling. Artificial Intelligence , 2004, 155(1): 1-39
doi: 10.1016/j.artint.2003.10.001
13 Elkan C. Naive bayesian learning. Technical Report CS97-557, Dept. of Computer Science and Engineering, UCSD , 1997
14 Blake C, Merz C J. UCI machine learning repository. http://www.ics.uci. edu/~mlearn/ MLRepository.html
15 Zadeh L A. Fuzzy logic= computing with words. IEEE Transactions on Fuzzy Systems , 1996, 4(2): 103-111
doi: 10.1109/91.493904
16 Zadeh L A. The concept of a linguistic variable and its application to approximate reasoning-I. Information Sciences , 1975, 8(3): 199-249
doi: 10.1016/0020-0255(75)90036-5
17 Sufyan Beg M, Thint M, Qin Z. Pnl-enhanced restricted domain question answering system. In: Proceedings of the 2007 IEEE International Fuzzy Systems Conference . 2007, 1-7
18 Qin Z, Thint M, Beg M S. Deduction engine design for pnl-based question answering system. In: Proceedings of the 12th International Fuzzy Systems Association World Congress . 2007, 253-262
19 Lawry J. Modeling and reasoning with vague concepts. Springer , 2006
20 Lawry J, Shanahan J G, Ralescu A. Modelling with words: learning, fusion, and reasoning within a formal linguistic representation framework. Volume 2873. Springer , 2003
21 Qin Z, Lawry J. Lfoil: linguistic rule induction in the label semantics framework. Fuzzy Sets and Systems , 2008, 159(4): 435-448
doi: 10.1016/j.fss.2007.10.008
22 Baldwin J F, Martin T P, Pilsworth B W. Fril-fuzzy and evidential reasoning in artificial intelligence. John Wiley & Sons, Inc. , 1995
23 Zhang W, Qin Z. Dissimilarity measure of logical expressions. In: Proceedings of the 2010 International Conference on Machine Learning and Cybernetics (ICMLC) . 2010, 199-203
doi: 10.1109/ICMLC.2010.5581066
24 Zhang W, Qin Z. Clustering data and imprecise concepts. In: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ) . 2011, 603-608
25 Jeffrey R C. The logic of decision. University of Chicago Press , 1990
26 Qin Z, Lawry J. Fuzziness and performance: an empirical study with linguistic decision trees. In: Proceedings of the 12th International Fuzzy Systems Association World Congress on Foundations of Fuzzy Logic and Soft Computing . 2007, 407-416
27 Randon N J, Lawry J. Classification and query evaluation using modelling with words. Information Sciences , 2006, 176(4): 438-464
doi: 10.1016/j.ins.2005.07.019
28 Qin Z. Naive bayes classification given probability estimation trees. In: Proceedings of the 5th International Conference on Machine Learning and Applications, ICMLA’06 . 2006, 34-42
29 Qin Z, Lawry J. Hybrid bayesian estimation trees based on label semantics. Lecture Notes in Computer Science , 2005, 896-907
doi: 10.1007/11518655_75
[1] Chaoqun LI,Liangxiao JIANG,Hongwei LI. Naive Bayes for value difference metric[J]. Front. Comput. Sci., 2014, 8(2): 255-264.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed