Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front Comput Sci    2012, Vol. 6 Issue (5) : 568-580    https://doi.org/10.1007/s11704-012-1175-2
RESEARCH ARTICLE
The ClasSi coefficient for the evaluation of ranking quality in the presence of class similarities
Anca Maria IVANESCU(), Marc WICHTERICH, Christian BEECKS, Thomas SEIDL
Data Management and Data Exploration Group, RWTH Aachen University, D-52056 Aachen, Germany
 Download: PDF(958 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Evaluationmeasures play an important role in the design of new approaches, and often quality is measured by assessing the relevance of the obtained result set.While many evaluation measures based on precision/recall are based on a binary relevance model, ranking correlation coefficients are better suited for multi-class problems. State-of-the-art ranking correlation coefficients like Kendall’s τ and Spearman’s ρ do not allow the user to specify similarities between differing object classes and thus treat the transposition of objects from similar classes the same way as that of objects from dissimilar classes. We propose ClasSi, a new ranking correlation coefficient which deals with class label rankings and employs a class distance function to model the similarities between the classes. We also introduce a graphical representation of ClasSi which describes how the correlation evolves throughout the ranking.

Keywords ranking      quality measure      class similarity      ClasSi     
Corresponding Author(s): IVANESCU Anca Maria,Email:ivanescu@informatik.rwth-aachen.de   
Issue Date: 01 October 2012
 Cite this article:   
Marc WICHTERICH,Christian BEECKS,Thomas SEIDL, et al. The ClasSi coefficient for the evaluation of ranking quality in the presence of class similarities[J]. Front Comput Sci, 2012, 6(5): 568-580.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-012-1175-2
https://academic.hep.com.cn/fcs/EN/Y2012/V6/I5/568
1 van Rijsbergen C J. Information Retrieval. 2nd ed. London: Butterworth-Heinemann, 1979
2 Manning C D, Raghavan P, Schütze H. Introduction to Information Retrieval. Cambridge: Cambridge University Press, 2008
doi: 10.1017/CBO9780511809071
3 Flach P A, Blockeel H, Ferri C, Hernández-Orallo J, Struyf J. Decision support for data mining; introduction to ROC analysis and its applications. In: Mladenic D, Lavra?n, Bohanec M, Moyle S, eds. Data Mining and Decision Support: Integration and Collaboration . Boston: Kluwer Academic Publishers, 2003, 81-90
4 Hand D J, Till R J. A simple generalization of the area under the ROC curve for multiple class classification problems. Machine Learning , 2001, 45(2): 171-186
doi: 10.1023/A:1010920819831
5 Ferri C, Hernández-Orallo J, Salido M A. Volume under the ROC surface for multi-class problems. In: Proceedings of the 14th European Conference on Machine Learning . 2003, 108-120
6 Hassan M R, Ramamohanarao K, Karmakar C K, Hossain M M, Bailey J. A novel scalable multi-class ROC for effective visualization and computation. In: Proceedings of the 14th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Part I . 2010, 107-120
7 Kendall M. A new measure of rank correlation. Biometrika , 1938, 30(1-2): 81-89
8 Spearman C. The proof and measurement of association between two things. The American Journal of Psychology , 1987, 100(3/4): 441-471
doi: 10.2307/1422689
9 Kendall M, Gibbons J D. Rank Correlation Methods. London: Edward Arnold, 1990
10 Goodman L A, Kruskal W H. Measures of association for cross classifications. Journal of the American Statistical Association , 1954, 49(268): 732-764
11 Somers R H. A new asymmetric measure of association for ordinal variables. American Sociological Review , 1962, 27(6): 799-811
doi: 10.2307/2090408
12 Ivanescu A, Wichterich M, Seidl T. ClasSi: measuring ranking quality in the presence of object classes with similarity information. In: Proceedings of PAKDD 2011 Quality Issues, Measures of Interestingness and Evaluation of Data Mining Models Workshop . 2011, 185-196
13 Beecks C, Uysal M S, Seidl T. Signature quadratic form distance. In: Proceedings of the 2010 ACM International Conference on Image and Video Retrieval . 2010, 438-445
doi: 10.1145/1816041.1816105
14 Rubner Y, Tomasi C, Guibas L J. The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision , 2000, 40(2): 99-121
doi: 10.1023/A:1026543900054
15 Wang J Z, Li J, Wiederhold G. Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2001, 23(9): 947-963
doi: 10.1109/34.955109
16 van de Sande K E A, Gevers T, Snoek C G M. Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2010, 32(9): 1582-1596
doi: 10.1109/TPAMI.2009.154
17 Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten I H. The WEKA data mining software: an update. ACM SIGKDD Explorations Newsletter , 2009, 11(1): 10-18
doi: 10.1145/1656274.1656278
[1] Yunyun WANG, Jiao HAN, Yating SHEN, Hui XUE. Pointwise manifold regularization for semi-supervised learning[J]. Front. Comput. Sci., 2021, 15(1): 151303-.
[2] Panthadeep BHATTACHARJEE, Pinaki MITRA. A survey of density based clustering algorithms[J]. Front. Comput. Sci., 2021, 15(1): 151308-.
[3] Qianchen YU, Zhiwen YU, Zhu WANG, Xiaofeng WANG, Yongzhi WANG. Estimating posterior inference quality of the relational infinite latent feature model for overlapping community detection[J]. Front. Comput. Sci., 2020, 14(6): 146323-.
[4] Parnika PARANJAPE, Meera DHABU, Parag DESHPANDE. A novel classifier for multivariate instance using graph class signatures[J]. Front. Comput. Sci., 2020, 14(4): 144307-.
[5] Muhammad Aminur RAHAMAN, Mahmood JASIM, Md. Haider ALI, Md. HASANUZZAMAN. Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language[J]. Front. Comput. Sci., 2020, 14(3): 143302-.
[6] Xibin DONG, Zhiwen YU, Wenming CAO, Yifan SHI, Qianli MA. A survey on ensemble learning[J]. Front. Comput. Sci., 2020, 14(2): 241-258.
[7] Hui XUE, Haiming XU, Xiaohong CHEN, Yunyun WANG. A primal perspective for indefinite kernel SVM problem[J]. Front. Comput. Sci., 2020, 14(2): 349-363.
[8] Ying LI, Xiangwei KONG, Haiyan FU, Qi TIAN. Contextual modeling on auxiliary points for robust image reranking[J]. Front. Comput. Sci., 2019, 13(5): 1010-1022.
[9] Yu HONG, Kai WANG, Weiyi GE, Yingying QIU, Guodong ZHOU. Cursor momentum for fascination measurement[J]. Front. Comput. Sci., 2019, 13(2): 396-412.
[10] Rizwan Ahmed KHAN, Alexandre MEYER, Hubert KONIK, Saida BOUAKAZ. Saliency-based framework for facial expression recognition[J]. Front. Comput. Sci., 2019, 13(1): 183-198.
[11] Changlong WANG, Zhiyong FENG, Xiaowang ZHANG, Xin WANG, Guozheng RAO, Daoxun FU. ComR: a combined OWL reasoner for ontology classification[J]. Front. Comput. Sci., 2019, 13(1): 139-156.
[12] Munish SAINI, Kuljit Kaur CHAHAL. Change profile analysis of open-source software systems to understand their evolutionary behavior[J]. Front. Comput. Sci., 2018, 12(6): 1105-1124.
[13] Xiaohui TAN, Yachun FAN, Ruiliang GUO. Local features and manifold ranking coupled method for sketch-based 3D model retrieval[J]. Front. Comput. Sci., 2018, 12(5): 1000-1012.
[14] Shuaiqiang WANG, Yilong YIN. Polygene-based evolutionary algorithms with frequent pattern mining[J]. Front. Comput. Sci., 2018, 12(5): 950-965.
[15] Chengyu WANG, Guomin ZHOU, Xiaofeng HE, Aoying ZHOU. NERank+: a graph-based approach for entity ranking in document collections[J]. Front. Comput. Sci., 2018, 12(3): 504-517.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed