Compositional metric learning for multi-label classification

doi:10.1007/s11704-020-9294-7

Front. Comput. Sci.

2021, Vol. 15

Issue (5) : 155320 https://doi.org/10.1007/s11704-020-9294-7

RESEARCH ARTICLE

Compositional metric learning for multi-label classification

Yan-Ping SUN^1,², Min-Ling ZHANG^1,^2,³(

)

¹. School of Computer Science and Engineering, Southeast University, Nanjing 210096, China
². Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China
³. Collaborative Innovation Center forWireless Communications Technology, Nanjing 211100, China

Download: PDF(577 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Multi-label classification aims to assign a set of proper labels for each instance, where distance metric learning can help improve the generalization ability of instance-based multi-label classification models. Existing multi-label metric learning techniques work by utilizing pairwise constraints to enforce that examples with similar label assignments should have close distance in the embedded feature space. In this paper, a novel distance metric learning approach for multi-label classification is proposed by modeling structural interactions between instance space and label space. On one hand, compositional distance metric is employed which adopts the representation of a weighted sum of rank-1 PSD matrices based on component bases. On the other hand, compositional weights are optimized by exploiting triplet similarity constraints derived from both instance and label spaces. Due to the compositional nature of employed distance metric, the resulting problem admits quadratic programming formulation with linear optimization complexity w.r.t. the number of training examples.We also derive the generalization bound for the proposed approach based on algorithmic robustness analysis of the compositional metric. Extensive experiments on sixteen benchmark data sets clearly validate the usefulness of compositional metric in yielding effective distance metric for multi-label classification.

Keywords machine learning multi-label learning metric learning compositionalmetric positive semidefinite matrix decomposition

Corresponding Author(s): Min-Ling ZHANG

Just Accepted Date: 04 March 2020 Issue Date: 31 December 2020

Cite this article:

Yan-Ping SUN,Min-Ling ZHANG. Compositional metric learning for multi-label classification[J]. Front. Comput. Sci., 2021, 15(5): 155320.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-020-9294-7
https://academic.hep.com.cn/fcs/EN/Y2021/V15/I5/155320

1	M L Zhang, Z H Zhou. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(8): 1819–1837 https://doi.org/10.1109/TKDE.2013.39
2	E Gibaja, S Ventura. A tutorial on multilabel learning. ACM Computing Surveys, 2015, 47(3): 52 https://doi.org/10.1145/2716262
3	F Briggs, B Lakshminarayanan, L Neal, X Z Fern, R Raich, S J Hadley, A S Hadley, M G Betts. Acoustic classification of multiple simultaneous bird species: a multi-instance multi-label approach. Journal of the Acoustical Society of America, 2012, 131(6): 4640–4650 https://doi.org/10.1121/1.4707424
4	R Cabral, F DelaTorre, J P Costeira, A Bernardino. Matrix completion for weakly-supervised multi-label image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 121–135 https://doi.org/10.1109/TPAMI.2014.2343234
5	J Liu, W C Chang, Y Wu, Y Yang. Deep learning for extreme multi-label text classification. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017, 115–124 https://doi.org/10.1145/3077136.3080834
6	X Pan, Y X Fan, J Jia, H B Shen. Identifying RNA-binding proteins using multi-label deep learning. Science China Information Sciences, 2019, 62: 19103 https://doi.org/10.1007/s11432-018-9558-2
7	L Sun, H Ge, W Kang. Non-negative matrix factorization based modeling and training algorithm for multi-label learning. Frontiers of Computer Science, 2019, 13(6): 1243–1254 https://doi.org/10.1007/s11704-018-7452-y
8	A Bellet, A Habrard, M Sebban. Metric learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 2015, 9(1): 1–151 https://doi.org/10.2200/S00626ED1V01Y201501AIM030
9	F Wang, J Sun. Survey on distance metric learning and dimensionality reduction in data mining. Data Mining and Knowledge Discovery, 2015, 29(2): 534–564 https://doi.org/10.1007/s10618-014-0356-z
10	W Liu, I W Tsang. Large margin metric learning for multi-label prediction. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2800–2806
11	H Goukand, B Pfahringer, M Cree. Learning distance metrics for multilabel classification. In: Proceedings of the 8th Asian Conference on Machine Learning. 2016, 318–333
12	Y Zhang, J Schneider. Maximum margin output coding. In: Proceedings of the 29th International Conference on Machine Learning. 2012, 1575–1582
13	Y Verma, C V Jawahar. Image annotation by propagating labels from semantic neighbourhoods. International Journal of Computer Vision, 2017, 121(1): 126–148 https://doi.org/10.1007/s11263-016-0927-0
14	H Gouk, B Pfahringer, M Cree. Learning similarity metrics by factorising adjacency matrices. 2015, arXiv preprint arXiv: 1511.06442
15	J Ni, J Liu, C Zhang, D Ye, Z Ma. Fine-grained patient similarity measuring using deep metric learning. In: Proceedings of the 26th ACM International Conference on Information and Knowledge Management. 2017, 1189–1198 https://doi.org/10.1145/3132847.3133022
16	Y Shi, A Bellet, F Sha. Sparse compositional metric learning. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 2078–2084
17	J St. Amand, J Huan. Sparse compositional local metric learning. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2017, 1097–1104 https://doi.org/10.1145/3097983.3098153
18	Z H Zhou, M L Zhang, S J Huang, Y F Li. Multi-instance multi-label learning. Artificial Intelligence, 2012, 176(1): 2291–2320 https://doi.org/10.1016/j.artint.2011.10.002
19	ML Zhang, L Wu. LIFT:multi-label learning with label-specific features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 107–120 https://doi.org/10.1109/TPAMI.2014.2339815
20	J Huang, G Li, Q Huang, X Wu. Learning label-specific features and class-dependent labels for multi-label classification. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(12): 3309–3323 https://doi.org/10.1109/TKDE.2016.2608339
21	K Q Weinberger, L K Saul. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research, 2009, 10: 207–244
22	S J Huang, Z H Zhou. Multi-label learning by exploiting label correlations locally. In: Proceedings of the 26th AAAI Conference on Artificial Intelligence. 2012, 949–955
23	Y Zhu, J Kwok, Z H Zhou. Multi-label learning with global and local correlation. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(6): 1081–1094 https://doi.org/10.1109/TKDE.2017.2785795
24	G X Yuan, C H Ho, C J Lin. An improved GLMNET for L1-regularized logistic regression. Journal of Machine Learning Research, 2012, 13: 1999–2030 https://doi.org/10.1145/2020408.2020421
25	A Beck, M Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. Siam Journal on Imaging Sciences, 2009, 2(1): 183–202 https://doi.org/10.1137/080716542
26	K C Toh, S Yun. An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pacific Journal of Optimization, 2010, 6(3): 615–640
27	A Bellet, A Habrard. Robustness and generalization for metric learning. Neurocomputing, 2015, 151(14): 259–267 https://doi.org/10.1016/j.neucom.2014.09.044
28	J Read, B Pfahringer, G Holmes, E Frank. Classifier chains for multi-label classification. Machine Learning, 2011, 85(3): 333–359 https://doi.org/10.1007/s10994-011-5256-5
29	M L Zhang, Z H Zhou. ML-kNN: a lazy learning approach to multi-label learning. Pattern Recognition, 2007, 40(7): 2038–2048 https://doi.org/10.1016/j.patcog.2006.12.019
30	J Rong, S Wang, Z H Zhou. Learning a distance metric from multiinstance multi-label data. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2009, 896–902 https://doi.org/10.1109/CVPRW.2009.5206684
31	Y Verma, C V Jawahar. A robust distance with correlated metric learning for multi-instance multi-label data. In: Proceedings of the 24th ACM International Conference on Multimedia. 2016, 441–445 https://doi.org/10.1145/2964284.2967259
32	M L Zhang, Y K Li, Y Y Liu, X Geng. Binary relevance for multi-label learning: an overview. Frontiers of Computer Science, 2018, 12(2): 191–202 https://doi.org/10.1007/s11704-017-7031-7
33	Y Wu, Y Lin, X Dong, Y Yan, W Bian, Y Yang. Progressive learning for person re-identification with one example. IEEE Transactions on Image Processing, 2019, 28(6): 2872–2881 https://doi.org/10.1109/TIP.2019.2891895
34	L Sun, S Ji, J Ye. Multi-label Dimensionality Reduction. London: Chapman and Hall/CRC, 2013
35	R B Pereira, A Plastino, B Zadrozny, L H C Merschmann. Categorizing feature selection methods for multi-label classification. Artificial Intelligence Review, 2018, 49(1): 57–78 https://doi.org/10.1007/s10462-016-9516-4
36	J Zhang, C Li, D Cao, Y Lin, S Su, L Dai, S Li. Multi-label learning with label-specific features by resolving label correlations. Knowledge-Based Systems, 2018, 159: 148–157 https://doi.org/10.1016/j.knosys.2018.07.003
37	Z S Chen, M L Zhang. Multi-label learning with regularization enriched label-specific features. In: Proceedings of the 11th Asian Conference on Machine Learning. 2019, 411–424
38	Y Yang, S Gopal. Multilabel classification with meta-level features in a learning-to-rank framework. Machine Learning, 2012, 88(1–2): 47–68 https://doi.org/10.1007/s10994-011-5270-7
39	S Canuto, M A Gonçalves, F Benevenuto. Exploiting new sentimentbased meta-level features for effective sentiment analysis. In: Proceedings of the 9th ACM International Conference on Web Search and Data Mining. 2016, 53–62 https://doi.org/10.1145/2835776.2835821
40	X Zhu, X Li, S Zhang. Block-row sparse multiview multilabel learning for image classification. IEEE Transactions on Cybernetics, 2016, 46(2): 450–461 https://doi.org/10.1109/TCYB.2015.2403356
41	C Zhang, Z Yu, Q Hu, P Zhu, X Liu, X Wang. Latent semantic aware multi-view multi-label classification. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 4414–4421
42	X Wu, Q G Chen, Y Hu, D B Wang, X Chang, X Wang, M L Zhang. Multiview multi-label learning with view-specific information extraction. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 3884–3890 https://doi.org/10.24963/ijcai.2019/539
43	R Zhang, F Nie, X Li, X Wei. Feature selection with multi-view data: a survey. Information Fusion, 2019, 50: 158–167 https://doi.org/10.1016/j.inffus.2018.11.019
44	Z H Zhou. Abductive learning: towards bridging machine learning and logical reasoning. Science China Information Sciences, 2019, 62: 076101 https://doi.org/10.1007/s11432-018-9801-4
45	Y Yang, Z Ma, A G Hauptmann, N Sebe. Feature selection for multimedia analysis by sharing information among multiple tasks. IEEE Transactions on Multimedia, 2013, 15(3): 661–669 https://doi.org/10.1109/TMM.2012.2237023
46	R Zhang, F Nie, X Li. Self-weighted supervised discriminative feature selection. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(8): 3913–3918 https://doi.org/10.1109/TNNLS.2017.2740341
47	R Zhang, F Nie, Y Wang, X Li. Unsupervised feature selection via adaptive multimeasure fusion. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(9): 2886–2892 https://doi.org/10.1109/TNNLS.2018.2884487

[1]

Highlights

Download

[1]	Zhen SONG, Yu GU, Zhigang WANG, Ge YU. DRPS: efficient disk-resident parameter servers for distributed machine learning[J]. Front. Comput. Sci., 2022, 16(4): 164321-.
[2]	Yu OU, Lang LI. Side-channel analysis attacks based on deep learning network[J]. Front. Comput. Sci., 2022, 16(2): 162303-.
[3]	Suyu MEI. A framework combines supervised learning and dense subgraphs discovery to predict protein complexes[J]. Front. Comput. Sci., 2022, 16(1): 161901-.
[4]	Xinyu TONG, Ziao YU, Xiaohua TIAN, Houdong GE, Xinbing WANG. Improving accuracy of automatic optical inspection with machine learning[J]. Front. Comput. Sci., 2022, 16(1): 161310-.
[5]	Yi REN, Ning XU, Miaogen LING, Xin GENG. Label distribution for multimodal machine learning[J]. Front. Comput. Sci., 2022, 16(1): 161306-.
[6]	Xia-an BI, Yiming XIE, Hao WU, Luyun XU. Identification of differential brain regions in MCI progression via clustering-evolutionary weighted SVM ensemble algorithm[J]. Front. Comput. Sci., 2021, 15(6): 156903-.
[7]	Xiaobing SUN, Tianchi ZHOU, Rongcun WANG, Yucong DUAN, Lili BO, Jianming CHANG. Experience report: investigating bug fixes in machine learning frameworks/libraries[J]. Front. Comput. Sci., 2021, 15(6): 156212-.
[8]	Jian SUN, Pu-Feng DU. Predicting protein subchloroplast locations: the 10th anniversary[J]. Front. Comput. Sci., 2021, 15(2): 152901-.
[9]	Syed Farooq ALI, Muhammad Aamir KHAN, Ahmed Sohail ASLAM. Fingerprint matching, spoof and liveness detection: classification and literature review[J]. Front. Comput. Sci., 2021, 15(1): 151310-.
[10]	Yuling MA, Chaoran CUI, Jun YU, Jie GUO, Gongping YANG, Yilong YIN. Multi-task MIML learning for pre-course student performance prediction[J]. Front. Comput. Sci., 2020, 14(5): 145313-.
[11]	Liang SUN, Hongwei GE, Wenjing KANG. Non-negative matrix factorization based modeling and training algorithm for multi-label learning[J]. Front. Comput. Sci., 2019, 13(6): 1243-1254.
[12]	Xu-Ying LIU, Sheng-Tao WANG, Min-Ling ZHANG. Transfer synthetic over-sampling for class-imbalance learning with limited minority class data[J]. Front. Comput. Sci., 2019, 13(5): 996-1009.
[13]	Yu-Feng LI, De-Ming LIANG. Safe semi-supervised learning: a brief introduction[J]. Front. Comput. Sci., 2019, 13(4): 669-676.
[14]	Wenhao ZHENG, Hongyu ZHOU, Ming LI, Jianxin WU. CodeAttention: translating source code to comments by exploiting the code constructs[J]. Front. Comput. Sci., 2019, 13(3): 565-578.
[15]	Hao SHAO. Query by diverse committee in transfer active learning[J]. Front. Comput. Sci., 2019, 13(2): 280-291.

Viewed

Full text

Abstract

Cited

Shared

Discussed