Please wait a minute...
Shopping Cart Email List Chinese
Advanced Search Fig/Tab
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Featured Articles  |  Most Downloaded  |  Most Reading
Online Submission Subscribe to Our RSS Content Feed
Front Comput Sci Chin    2009, Vol. 3 Issue (1) : 109-122    https://doi.org/10.1007/s11704-009-0013-7
RESEARCH ARTICLE
Incorporating prior knowledge into learning by dividing training data
Baoliang LU1,2(), Xiaolin WANG1, Masao UTIYAMA3
1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China; 2. MOE-Microsoft Key Lab for Intelligent Computing and Intelligent Systems, Shanghai Jiao Tong University, Shanghai 200240, China; 3. National Institute of Information and Communications Technology (NICT), Kyoto 619-0288, Japan
 Download: PDF(1519 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

In most large-scale real-world pattern classification problems, there is always some explicit information besides given training data, namely prior knowledge, with which the training data are organized. In this paper, we proposed a framework for incorporating this kind of prior knowledge into the training of min-max modular (M3) classifier to improve learning performance. In order to evaluate the proposed method, we perform experiments on a large-scale Japanese patent classification problem and consider two kinds of prior knowledge included in patent documents: patent’s publishing date and the hierarchical structure of patent classification system. In the experiments, traditional support vector machine (SVM) and M3-SVM without prior knowledge are adopted as baseline classifiers. Experimental results demonstrate that the proposed method is superior to the baseline classifiers in terms of training cost and generalization accuracy. Moreover,M3-SVM with prior knowledge is found to be much more robust than traditional support vector machine to noisy dated patent samples, which is crucial for incremental learning.
Keywords prior knowledge      patent classification      support vector machine      min-max modular network      task decomposition     
Corresponding Author(s): LU Baoliang,Email:bllu@sjtu.edu.cn, mutiyama@nict.go.jp   
Issue Date: 05 March 2009
 Cite this article:   
Baoliang LU,Xiaolin WANG,Masao UTIYAMA. Incorporating prior knowledge into learning by dividing training data[J]. Front Comput Sci Chin, 2009, 3(1): 109-122.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-009-0013-7
https://academic.hep.com.cn/fcs/EN/Y2009/V3/I1/109
1 Liu B, Li X L, Lee W S, Yu P S. Text classification by labeling words. AAAI , 2004
2 Wu X Y, Srihari R. Incorporating prior knowledge with weighted margin support vector machines. In: Proceedings of International Conference on Knowledge Discovery and Data Mining , 2004, 326-333
3 Schapire R E, Rochery M, Rabim M, Gupta N. Boosting with prior knowledge for call classification. IEEE Transactions on Speech and Audio Processing , 2005, 13, 174-181
doi: 10.1109/TSA.2004.840937
4 Zhu J B, Chen W L. Improving text categorization using domain knowledge In: Proceedings of International Conference on Applications of Natural Language to Information Systems , 2005, 103-113
5 Dayanik A, Lewis D D, Madigan D, Menkov V, Genkin A. Constructing informative prior distributions from domain knowledge in text classification. In: Proceedings of ACM’S Special Interest Group on Information Retrieval , 2006
6 Lu B L, Ito M. Task decomposition based on class relations: a modular neural network architecture for pattern classification. Biological and Artificial Computation: From Neuroscience to Technology. Springer, LNCS , 1997, 1240: 330-339
7 Lu B L, Ito M. Task decomposition and module combination based on class relations: A modular neural network for pattern classification. IEEE Transactions on Neural Networks , 1999, 10: 1244-1256
doi: 10.1109/72.788664
8 Anand R, Mehrotra K G, Mohan C K, Ranka S. An improved algorithm for neural network classification of imbalanced training sets. IEEE Transaction on Neural Netwook , 1993, 4: 962-969
doi: 10.1109/72.286891
9 Lu B L,Wang K A, Utiyama M, Isahara H. A part-versus-part method for massively parallel training of support vector machines. In: Proceedings of International Joint Conference on Neural Networks , 2004, 735-740
10 Krier M, Zaccá F. Automatic categorization applications at the European patent office. World Patent Information. Elsevier , 2002, 24(3): 187-196
11 Larkey L. Some issues in the automatic classification of US patents. Learning for Text Categorization. Technical Report WS-98-05 , 1998, 87-90
12 Larkey L. A patent search and classification system. In: Proceedings of the fourth ACM conference on Digital libraries , 1999, 179-187
13 Mase H, Tsuji H, Kinukawa H, Ishihara M. Automatic patents categorization and its evaluation. Transactions of Information Processing Society of Japan(IPSJ) , 1998
14 Fall C J, Benzineb K. Literature survey: Issues to be considered in the automatic classification of patents. World Intellectual Property Organization , 2002, 29
15 Fall C J, Torcsvári A, Benzineb K, Karetka G. Automated categorization in the international patent classification. In: Proceedings of ACM’S Special Interest Group on Information Retrieval . New York: ACM Press, 2003, 37: 10-25
16 Fujii A, Iwayama M, Kando N. Test collections for patent retrieval and patent classification in the 5th NTCIR workshop. In: Proceedings of the 5th international conference on language resources and evaluation , 2004, 1643-1646
17 Fujii A, Iwayama M, Kando N. Introduction to the special issue on patent processing. Information Processing and Management , 2007, 1149-1153
doi: 10.1016/j.ipm.2006.11.004
18 Wen Y M, Lu B L, Zhao H. Equal clustering makes min-max modular support vector machine more efficient. In: Proceedings of International Conference on Neural Information Processing , 2005, 77-82
19 Lian H C, Lu B L, Takikawa E, Hosoi S. Gender recognition using a min-max modular support vector machine. In: Proceedings of International Conference on Natural Computation , 2005, 438-441
20 Yang Y M, Pedersen J O. A comparattive study on feature selection in text categorization. In: Proceedings of International Conference on Machine Learning , 1997, 187-196
21 Sebastiani F. Machine learning in automated text categorization. ACM Computing Surveys , 2002, 34: 1-47
doi: 10.1145/505282.505283
22 Zhao H, Lu B L. A modular k-nearest neighbor classification method for massively parallel text categorization. In: Proceedings of First International Symposium on Computational and Information Science. Springer, LNCS , 2004, 3314: 867-872
23 Wu K, Lu B L, Uchiyama M, Isahara H. An empirical comparison of min-max-modular k-NN with different voting methods to largescale text categorization. Soft Computing - A Fusion of Foundations, Methodologies and Applications , 2008, 12(7): 647-655
24 Joachims T. Making large-scale support vector machine learning practical. Advances in Kernel Methods: Support Vector Learning . Cambridge: MIT Press, 1998
25 Lewis D D, Yang Y, Rose T, Li F. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research , 2004, 5: 361-397
26 Liu W, Xue G R, Yu Y, Zeng H J. Importance-based web page classification using cost-sensitive SVM. In: Proceedings of International Conference on Web-Age Information Management , 2005, 127-137
[1] Hui XUE, Haiming XU, Xiaohong CHEN, Yunyun WANG. A primal perspective for indefinite kernel SVM problem[J]. Front. Comput. Sci., 2020, 14(2): 349-363.
[2] Hui XUE, Sen LI, Xiaohong CHEN, Yunyun WANG. A maximum margin clustering algorithm based on indefinite kernels[J]. Front. Comput. Sci., 2019, 13(4): 813-827.
[3] Xu YU,Jing YANG,Zhiqiang XIE. Training SVMs on a bound vectors set based on Fisher projection[J]. Front. Comput. Sci., 2014, 8(5): 793-806.
[4] Shangfei WANG,Shan HE,Yue WU,Menghua HE,Qiang JI. Fusion of visible and thermal images for facial expression recognition[J]. Front. Comput. Sci., 2014, 8(2): 232-242.
[5] Lean YU, Shouyang WANG, Kin Keung LAI. Developing an SVM-based ensemble learning system for customer risk identification collaborating with customer relationship management[J]. Front Comput Sci Chin, 2010, 4(2): 196-203.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed