|
|
Pointwise manifold regularization for semi-supervised learning |
Yunyun WANG1( ), Jiao HAN1, Yating SHEN1, Hui XUE2 |
1. Department of Computer Science and Engineering, Nanjing University of Posts & Telecommunications, Nanjing 210046, China 2. School of Computer Science and Engineering, Southeast University, Nanjing 210096, China |
|
|
Abstract Manifold regularization (MR) provides a powerful framework for semi-supervised classification using both the labeled and unlabeled data. It constrains that similar instances over the manifold graph should share similar classification outputs according to the manifold assumption. It is easily noted that MR is built on the pairwise smoothness over the manifold graph, i.e., the smoothness constraint is implemented over all instance pairs and actually considers each instance pair as a single operand. However, the smoothness can be pointwise in nature, that is, the smoothness shall inherently occur “everywhere” to relate the behavior of each point or instance to that of its close neighbors. Thus in this paper, we attempt to develop a pointwise MR (PW_MR for short) for semi-supervised learning through constraining on individual local instances. In this way, the pointwise nature of smoothness is preserved, and moreover, by considering individual instances rather than instance pairs, the importance or contribution of individual instances can be introduced. Such importance can be described by the confidence for correct prediction, or the local density, for example. PW_MR provides a different way for implementing manifold smoothness. Finally, empirical results show the competitiveness of PW_MR compared to pairwise MR.
|
Keywords
semi-supervised classification
manifold regularization
pairwise smoothness
pointwise smoothness
local density
|
Corresponding Author(s):
Yunyun WANG
|
Just Accepted Date: 04 March 2020
Issue Date: 10 October 2020
|
|
1 |
Z H Zhou, M Li. Semi-supervised learning by disagreement. Knowledge and Information Systems, 2010, 24(3): 415–439
https://doi.org/10.1007/s10115-009-0209-z
|
2 |
X J Zhu, A B Goldberg. Introduction to semi-supervised learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 2009, 3(1): 1–130
https://doi.org/10.2200/S00196ED1V01Y200906AIM006
|
3 |
X J Zhu. Semi-supervised learning literature survey. Technical Report, 2005
|
4 |
O Chapelle, B Schölkopf, A Zien. Semi-supervised Learning. Cambridge, MA: MIT Press, 2006
https://doi.org/10.7551/mitpress/9780262033589.001.0001
|
5 |
P K Mallapragada, R Jin, A K Jain, Y Liu. Semiboost: boosting for semisupervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 31(11): 2000–2014
https://doi.org/10.1109/TPAMI.2008.235
|
6 |
T Joachims. Transductive inference for text classification using support vector machines. In: Proceedings of the 16th Annual International Conference on Machine Learning. 1999, 200–209
|
7 |
G Fung, O L Mangasarian. Semi-supervised support vector machines for unlabeled data classification. Optimization Methods and Software, 2001, 15(1): 29–44
https://doi.org/10.1080/10556780108805809
|
8 |
R Collobert, F Sinz, J Weston, L Bottou. Large scale transductive SVMs. Journal of Machine Learning Research, 2006, 7(8): 1687–1712
|
9 |
Y F Li, J T Kwok, Z H Zhou. Semi-supervised learning using label mean. In: Proceedings of the 26th Annual International Conference on Machine Learning. 2009, 633–640
https://doi.org/10.1145/1553374.1553456
|
10 |
Y Bengio, O Delalleau, N L Roux. Label propagation and quadratic criterion. In: Chapelle O, Schölkopf B, Zien A, eds. Semi-supervised Learning. Cambridge, MA: MIT Press, 2006, 193–216
|
11 |
X J Zhu, Z Ghahramani. Learning from labeled and unlabeled data with label propagation. Technical Report, 2002
|
12 |
A Blum, S Chawla. Learning from labeled and unlabeled data using graph mincuts. In: Proceedings of the 18th Annual International Conference on Machine Learning. 2001, 19–26
|
13 |
M Belkin, P Niyogi, V Sindhwani. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 2006, 7(11): 2399–2434
|
14 |
K Chen, S H Wang. Semi-supervised learning via regularized boosting working on multiple semi-supervised assumptions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(1): 129–143
https://doi.org/10.1109/TPAMI.2010.92
|
15 |
X F He. Laplacian regularized D-optimal design for active learning and its application to image retrieval. IEEE Transactions on Image Processing, 2009, 19(1): 254–263
https://doi.org/10.1109/TIP.2009.2032342
|
16 |
J Abernethy, O Chapelle, C Castillo. Web spam identification through content and hyperlinks. In: Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web. 2008, 41–44
https://doi.org/10.1145/1451983.1451994
|
17 |
Y Fang, K C C Chang, H W Lauw. Graph-based semi-supervised learning: realizing pointwise smoothness probabolistically. In: Proceedings of the 31st Annual International Conference on Machine Learning. 2014, 406–414
|
18 |
A Singh, R Nowak, J Zhu. Unlabeled data: now it helps, now it doesn’t. In: Proceedings of Advances in Neural Information Processing Systems. 2009, 1513–1520
|
19 |
L Wasserman, J D Lafferty. Statistical analysis of semi-supervised regression. In: Proceedings of Advances in Neural Information Processing Systems. 2008, 801–808
|
20 |
P Rigollet. Generlization error bounds in semi-supervised classification under the cluster assumption. Journal of Machine Learning Research, 2007, 8(7): 1369–1392
|
21 |
H Wang, S B Wang, Y F Li. Instance selection method for improving graph-based semi-supervised learning. Frontiers of Computer Science, 2018, 12(4): 725–735
https://doi.org/10.1007/s11704-017-6543-5
|
22 |
H Gan, Z Li, W Wu, Z Luo, R Huang. Safety-aware graph-based semisupervised learning. Expert Systems with Applications, 2018, 107: 243–254
https://doi.org/10.1016/j.eswa.2018.04.031
|
23 |
Y Wang, Y Meng, Y Li, S C Chen, Z Y Fu, H Xue. Semi-supervised manifold regularization with adaptive graph construction. Pattern Recognition Letters, 2017, 98: 90–95
https://doi.org/10.1016/j.patrec.2017.09.004
|
24 |
H T Gan, Z Z Luo, Y Sun, X G Xi, N Sang, R Huang. Towards designing risk-based safe laplacian regularized least squares. Expert Systems with Applicaions, 2016, 45: 1–7
https://doi.org/10.1016/j.eswa.2015.09.017
|
25 |
M H Quang, L Bazzani, V Murino. A unifying framework for vectorvalued manifold regularization and multi-view learning. In: Proceedings of the 30th Annual International Conference on Machine Learning. 2013, 100–108
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|