Label distribution learning (LDL) is a new learning paradigm to deal with label ambiguity and many researches have achieved the prominent performances. Compared with traditional supervised learning scenarios, the annotation with label distribution is more expensive. Direct use of existing active learning (AL) approaches, which aim to reduce the annotation cost in traditional learning, may lead to the degradation of their performance. To deal with the problem of high annotation cost in LDL, we propose the active label distribution learning via kernel maximum mean discrepancy (ALDL-kMMD) method to tackle this crucial but rarely studied problem. ALDL-kMMD captures the structural information of both data and label, extracts the most representative instances from the unlabeled ones by incorporating the nonlinear model and marginal probability distribution matching. Besides, it is also able to markedly decrease the amount of queried unlabeled instances. Meanwhile, an effective solution is proposed for the original optimization problem of ALDL-kMMD by constructing auxiliary variables. The effectiveness of our method is validated with experiments on the real-world datasets.
X Geng . Label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 2016, 28( 7): 1734–1748
2
M L, Zhang Z H Zhou . A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 2014, 26( 8): 1819–1837
3
X, Geng C, Yin Z H Zhou . Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35( 10): 2401–2412
4
B B, Gao H Y, Zhou J, Wu X Geng . Age estimation using expectation of label distribution learning. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 712–718
5
S G, Kong R O Mbouna . Head pose estimation from a 2D face image using 3D face morphing with depth parameters. IEEE Transactions on Image Processing, 2015, 24( 6): 1801–1808
6
Y, Zhou H, Xue X Geng . Emotion distribution recognition from facial expressions. In: Proceedings of the 23rd ACM International Conference on Multimedia. 2015, 1247–1250
7
D, Zhou X, Zhang Y, Zhou Q, Zhao X Geng . Emotion distribution learning from texts. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 638–647
8
X, Dong S, Gu W, Zhuge T, Luo C Hou . Active label distribution learning. Neurocomputing, 2021, 436: 12–21
9
C J C Burges . A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998, 2( 2): 121–167
10
A, McCallum K Nigam . A comparison of event models for naive Bayes text classification. In: Proceedings of AAAI-98 Workshop on Learning for Text Categorization. 1998, 41–48
11
S, Tong D Koller . Support vector machine active learning with applications to text classification. The Journal of Machine Learning Research, 2002, 2: 45–66
12
Y, Freund H S, Seung E, Shamir N Tishby . Selective sampling using the query by committee algorithm. Machine Learning, 1997, 28( 2–3): 133–168
13
Y, Guo D Schuurmans . Discriminative batch mode active learning. In: Proceedings of the 20th International Conference on Neural Information Processing Systems. 2007, 593–600
14
T, Ren X, Jia W, Li S Zhao . Label distribution learning with label correlations via low-rank approximation. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 3325–3331
15
A L, Berger V J D, Pietra S A D Pietra . A maximum entropy approach to natural language processing. Computational Linguistics, 1996, 22( 1): 39–71
16
S D, Pietra V D, Pietra J Lafferty . Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19( 4): 380–393
17
J, Nocedal S J Wright . Numerical Optimization. New York: Springer, 2006
18
N, Xu Y P, Liu X Geng . Label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 2021, 33( 4): 1632–1643
19
Wang J, Geng X, Xue H. Re-weighting large margin label distribution learning for classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, DOI: 10.1109/TPAMI.2021.3082623
20
Seung H S, Opper M, Sompolinsky H. Query by committee. In: Proceedings of the 5th Annual Workshop on Computational Learning Theory. 1992, 287–294
21
D D, Lewis J Catlett . Heterogeneous uncertainty sampling for supervised learning. In: Cohen W W, ed. Machine Learning Proceedings. New Brunswick: Elsevier, 1994
22
M F, Balcan A, Broder T Zhang . Margin based active learning. In: Proceedings of the 20th International Conference on Computational Learning Theory. 2007, 35–50
23
D V Lindley . On a measure of the information provided by an experiment. The Annals of Mathematical Statistics, 1956, 27( 4): 986–1005
24
K, Yu J, Bi V Tresp . Active learning via transductive experimental design. In: Proceedings of the 23rd International Conference on Machine Learning. 2006, 1081–1088
25
Nguyen H T, Smeulders A. Active learning using pre-clustering. In: Proceedings of the 21st International Conference on Machine Learning. 2004, 9
26
F, Nie D, Xu X Li . Initialization independent clustering with actively self-training method. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42( 1): 17–27
27
D, Cai X He . Manifold adaptive experimental design for text categorization. IEEE Transactions on Knowledge and Data Engineering, 2012, 24( 4): 707–719
28
S J, Huang R, Jin Z H Zhou . Active learning by querying informative and representative examples. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36( 10): 1936–1949
29
Huang S J, Zhou Z H. Active query driven by uncertainty and diversity for incremental multi-label learning. In: Proceedings of the 13th IEEE International Conference on Data Mining. 2013, 1079–1084
30
S J, Huang J W, Zhao Z Y Liu . Cost-effective training of deep CNNs with active model adaptation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 1580–1588
31
S J, Huang J L, Chen X, Mu Z H Zhou . Cost-effective active learning from diverse labelers. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 1879–1885
32
Yang Y, Zhou D W, Zhan D C, Xiong H, Jiang Y, Yang J. Cost-effective incremental deep model: matching model capacity with the least sampling. IEEE Transactions on Knowledge and Data Engineering, 2021, DOI: 10.1109/TKDE.2021.3132622
33
Y P, Tang S J Huang . Dual active learning for both model and data selection. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence. 2021, 3052–3058
34
K M, Borgwardt A, Gretton M J, Rasch H P, Kriegel B, Schölkopf A J Smola . Integrating structured biological data by Kernel Maximum Mean Discrepancy. Bioinformatics, 2006, 22( 14): e49–e57
35
R, Fortet E Mourier . Convergence de la répartition empirique vers la répartition théorique. Annales Scientifiques de l'École Normale Supérieure, 1953, 70( 3): 267–285
36
M B, Eisen P T, Spellman P O, Brown D Botstein . Cluster analysis and display of genome-wide expression patterns. Proceedings of the National Academy of Sciences of the United States of America, 1998, 95( 25): 14863–14868
37
A, Kapoor K, Grauman R, Urtasun T Darrell . Gaussian processes for object categorization. International Journal of Computer Vision, 2010, 88( 2): 169–188
38
F B, Guo Y Lin . Identify protein-coding genes in the genomes of Aeropyrum pernix K1 and Chlorobium tepidum TLS. Journal of Biomolecular Structure and Dynamics, 2009, 26( 4): 413–420
39
Z, Ghafoori J C, Bezdek C, Leckie S Karunasekera . Unsupervised and active learning using maximin-based anomaly detection. In: Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2019, 90–106
