Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front. Comput. Sci.    2014, Vol. 8 Issue (5) : 807-815    https://doi.org/10.1007/s11704-014-3317-1
RESEARCH ARTICLE
Dimensionality reduction via kernel sparse representation
Zhisong PAN(),Zhantao DENG,Yibing WANG,Yanyan ZHANG
College of Command Information Systems, PLA University of Science and Technology, Nanjing 210007, China
 Download: PDF(392 KB)  
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Dimensionality reduction (DR) methods based on sparse representation as one of the hottest research topics have achieved remarkable performance in many applications in recent years. However, it’s a challenge for existing sparse representation based methods to solve nonlinear problem due to the limitations of seeking sparse representation of data in the original space. Motivated by kernel tricks, we proposed a new framework called empirical kernel sparse representation (EKSR) to solve nonlinear problem. In this framework, nonlinear separable data are mapped into kernel space in which the nonlinear similarity can be captured, and then the data in kernel space is reconstructed by sparse representation to preserve the sparse structure, which is obtained by minimizing a ?1 regularization-related objective function. EKSR provides new insights into dimensionality reduction and extends two models: 1) empirical kernel sparsity preserving projection (EKSPP), which is a feature extraction method based on sparsity preserving projection (SPP); 2) empirical kernel sparsity score (EKSS), which is a feature selection method based on sparsity score (SS). Both of the two methods can choose neighborhood automatically as the natural discriminative power of sparse representation. Compared with several existing approaches, the proposed framework can reduce computational complexity and be more convenient in practice.

Keywords feature extraction      feature selection      sparse representation      kernel trick     
Corresponding Author(s): Zhisong PAN   
Issue Date: 11 October 2014
 Cite this article:   
Zhisong PAN,Zhantao DENG,Yibing WANG, et al. Dimensionality reduction via kernel sparse representation[J]. Front. Comput. Sci., 2014, 8(5): 807-815.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-014-3317-1
https://academic.hep.com.cn/fcs/EN/Y2014/V8/I5/807
1 Marcellin M, Gormish M, Bilgin A, Boliek M. An overview of JPEG-2000. In: Proceedings of the 2000 IEEE Data Compression Conference. 2000, 523-541
https://doi.org/10.1109/DCC.2000.838192
2 Elad M, Aharon M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Process, 2006, 15(12): 3736-3745
https://doi.org/10.1109/TIP.2006.881969
3 Marial J, Bach F, Ponce J, Sapiro J, Zisserman A. Non-local sparse models for image restoration. In: Proceedings of the 12th IEEE International Conference on Computer Vision. 2009, 2272-2279
4 Huang K, Aviyente S. Sparse representation for signal classification. In: Proceedings of Advances in Neural Information Processing Systems. 2006, 609-616
5 Davenport M, Duarte M, Wakin M, Takhar D, Kelly K, Baraniuk R. The smashed filter for compressive classification and target recognition. In: Proceedings of IS&T/SPIE Symposium on Electronic Imaging: Computational Imaging. 2007, 64980H-64980H-12
6 Donoho D. For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution. Communications On Pure and Applied Mathematics, 2006, 59(6): 797-829
https://doi.org/10.1002/cpa.20132
7 Scholkopf B, Smola A, Muller K R. Kernel, principal component analysis. In: Proceedings of the 1997 International Conference on Artificial Neural Networks. 1997, 583-588
8 Qiao L, Chen S, Tan X. Sparsity preserving projections with applications to face recognition. Pattern Recognition, 2010, 43(1): 331-341
https://doi.org/10.1016/j.patcog.2009.05.005
9 Turk M, Pentland A. Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 1991, 3(1): 71-86
https://doi.org/10.1162/jocn.1991.3.1.71
10 He X F, Niyogi P. Locality preserving projections. In: Proceedings of Advances in Neural Information Processing Systems. 2003, 16: 234-241
11 Yang Y, Nie F, Xiang S, Zhuang Y, Wang W. Local and global regressive mapping for manifold learning with out-of-sample extrapolation. In: Proceedings of the 24th AAAI Conference on Artificial Intelligence. 2010, 649-654
12 Belhumeur P N, Hespanha J P, Kriegman D J. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. In: Proceedings of the 1997 IEEE Transactions on Pattern Analysis and Machine Intelligence. 1997, 19(7): 711-720
https://doi.org/10.1109/34.598228
13 Xu D, Yan S C, Tao D C, Lin S, Zhang H J. Marginal fisher analysis and its variants for human gait recognition and content based image retrieval. In: Proceedings of the 2007 IEEE Transactions on Image Processing. 2007, 16(11): 2811-2821
https://doi.org/10.1109/TIP.2007.906769
14 Li H F, Jiang T, Zhang K S. Efficient and robust feature extraction by maximum margin criterion. In: Proceedings of the 2006 IEEE Transactions on Neural Networks. 2006, 17(1): 157-165
https://doi.org/10.1109/TNN.2005.860852
15 Liu J, Chen S C, Tan X Y, Zhang D Q. Comments on “Efficient and robust feature extraction by maximum margin criterion”. In: Proceedings of the 2007 IEEE Transactions on Neural Networks. 2007, 18(6): 1862-1864
https://doi.org/10.1109/TNN.2007.900813
16 Zhang D Q, Zhou Z H, Chen S C. Semi-supervised dimensionality reduction. In: Proceedings of the 2007 International Conference on Data Mining. 2007, 629-634
https://doi.org/10.1137/1.9781611972771.73
17 Cai D, He X F, Han J W. Semi-supervised discriminant analysis. In: Proceedings of the 11th IEEE International Conference on Computer Vision. 2007, 1-7
18 Sugiyama M, Ide T, Nakajima S, Sese J. Semi-supervised local fisher discriminant analysis for dimensionality reduction. Machine Learning, 2008, 78(1-2): 35-61
19 Qiao L, Zhang L, Chen S. Dimensionality reduction with adaptive graph. Frontiers of Computer Science, 2013, 7(5): 745-753
https://doi.org/10.1007/s11704-013-2234-z
20 Bishop C M. Neural Networks for Pattern Recognition. Oxford: Oxford University Press, 1995
21 He X, Cai D, Niyogi P. Laplacian score for feature selection. In: Proceedings of the Advances in Neural Information Processing Systems. 2005, 17: 507-514
22 Liu M X, Sun D, Zhang D Q. Sparsity score: a new filter feature selection method based on L1 graph. In: Proceedings of the 21st International Conference on Pattern Recognition. 2012, 11-15
23 Yang Y, Ma Z, Hauptmann A, Sebe N. Feature selection for multimedia analysis by sharing information among multiple tasks. In: Proceedings of the 2013 IEEE Transactions on Multimedia. 2013, 15(3): 661-669
https://doi.org/10.1109/TMM.2012.2237023
24 Ma Z , Nie F P, Yang Y, Uijlings J R R, Sebe N. Web image annotation via subspace-sparsity collaborated feature selection. In: Proceedings of the 2012 IEEE Transactions on Multimedia. 2012, 14(4): 1021-1030
https://doi.org/10.1109/TMM.2012.2187179
25 Yang Y, Shen H T, Ma Z, Huang Z, Zhou X F. ?2,1-norm regularized discriminative feature selection for unsupervised learning. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence. 2011(2): 1589-1594
26 Zhang D, Chen S, Zhou Z. Constraint score: a new filter method for feature selection with pairwise constraints. Pattern Recognition, 2008, 41(5): 1440-1451
https://doi.org/10.1016/j.patcog.2007.10.009
27 Zhao Z, Liu H. Semi-supervised feature selection via spectral analysis. In: Proceedings of the 7th SIAM International Conference on Data Mining. 2007, 641-646
28 Gao S, Tsang I W H, Chia L T. Kernel sparse representation for image classification and face recognition. In: Proceedings of the 11th European Conference on Computer Vision. 2010, 1-14
29 Zhang L, Zhou W, Chang P, Liu J, Yan Z, Wang T, Li F. Kernel sparse representation-based classifier. In: Proceedings of the 2012 IEEE Transactions on Signal Processing. 2012, 1684-1695
30 Yin J, Liu Z, Jin Z, Yang W. Kernel sparse representation based classification. Neurocomputing, 2012, 77(1): 120-128
https://doi.org/10.1016/j.neucom.2011.08.018
31 Chen Y, Nasser N M, Tran T D. Hyperspectral image classification via kernel sparse representation. In: Proceedings of the 2013 IEEE Transactions on Geoscience and Remote Senseing. 2013, 51(1): 217-231
32 Xiong H, Swamy M N S, Ahmad M O. Optimizing the kernel in the empirical feature space. In: Proceedings of the 2005 IEEE Transactions on Neural Networks. 2005, 16: 460-474
https://doi.org/10.1109/TNN.2004.841784
33 Wang Z, Chen S, Sun T. MultiK-MHKS: a novel multiple kernel learning algorithm. In: Proceedings of the 2008 IEEE Transactions on Pattern Analysis and Machine Intelligence. 2008, 30(2): 348-353
https://doi.org/10.1109/TPAMI.2007.70786
34 Donoho D. Compressed sensing. In: Proceedings of the 2006 IEEE Transactions on Information Theory. 2006, 52(4): 1289-1306
https://doi.org/10.1109/TIT.2006.871582
35 Shawe-Taylor J, Cristianini N. Kernel Methods for Pattern Analysis. Cambridge: Cambridge University Press, 2004
https://doi.org/10.1017/CBO9780511809682
36 Martinez M, Kak A C. PCA versus LDA. In: Proceedings of the 2001 IEEE Transactions on Pattern Analysis and Machine Intelligence. 2001, 23(2): 228-233
https://doi.org/10.1109/34.908974
37 Wright J, Yang A, Sastry S, Ma Y. Robust face recognition via sparse representation. In: Proceedings of the 2009 IEEE Transactions on Pattern Analysis and Machine Intelligence. 2009, 31(2): 210-227
https://doi.org/10.1109/TPAMI.2008.79
38 Tsang I, Kocsor A, Kwok J. Efficient kernel feature extraction for massive data sets. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and DataMining. 2006, 724-729
https://doi.org/10.1145/1150402.1150494
39 Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2006, 68(1): 49-67
https://doi.org/10.1111/j.1467-9868.2005.00532.x
40 Liu J, Ye J. Moreau-yosida regularization for grouped tree structure learning. In: Proceedings of Advances in Neural Information Processing Systems. 2010, 23: 1459-1467
41 Jacob L, Obozinski G, Vert J P. Group lasso with overlap and graph lasso. In: Proceedings of the 26th ACM International Conference on Machine Learning. 2009, 433-440
[1] Xia-an BI, Yiming XIE, Hao WU, Luyun XU. Identification of differential brain regions in MCI progression via clustering-evolutionary weighted SVM ensemble algorithm[J]. Front. Comput. Sci., 2021, 15(6): 156903-.
[2] Zhiyong YU, Xiangping ZHENG, Fangwan HUANG, Wenzhong GUO, Lin SUN, Zhiwen YU. A framework based on sparse representation model for time series prediction in smart city[J]. Front. Comput. Sci., 2021, 15(1): 151305-.
[3] Lei CHEN, Kai SHAO, Xianzhong LONG, Lingsheng WANG. Multi-task regression learning for survival analysis via prior information guided transductive matrix completion[J]. Front. Comput. Sci., 2020, 14(5): 145312-.
[4] Xuejun WANG, Feilong CAO, Wenjian WANG. Adaptive sparse and dense hybrid representation with nonconvex optimization[J]. Front. Comput. Sci., 2020, 14(4): 144306-.
[5] Parnika PARANJAPE, Meera DHABU, Parag DESHPANDE. A novel classifier for multivariate instance using graph class signatures[J]. Front. Comput. Sci., 2020, 14(4): 144307-.
[6] Ratha PECH, Dong HAO, Hong CHENG, Tao ZHOU. Enhancing subspace clustering based on dynamic prediction[J]. Front. Comput. Sci., 2019, 13(4): 802-812.
[7] Farid FEYZI, Saeed PARSA. Inforence: effective fault localization based on information-theoretic analysis and statistical causal inference[J]. Front. Comput. Sci., 2019, 13(4): 735-759.
[8] Nannan XIE, Xing WANG, Wei WANG, Jiqiang LIU. Fingerprinting Android malware families[J]. Front. Comput. Sci., 2019, 13(3): 637-646.
[9] Yaru XIAN, Jun XIAO, Ying WANG. A fast registration algorithm of rock point cloud based on spherical projection and feature extraction[J]. Front. Comput. Sci., 2019, 13(1): 170-182.
[10] Xuegang HU, Peng ZHOU, Peipei LI, Jing WANG, Xindong WU. A survey on online feature selection with streaming features[J]. Front. Comput. Sci., 2018, 12(3): 479-493.
[11] Wei SHAO,Yi DING,Hong-Bin SHEN,Daoqiang ZHANG. Deep model-based feature extraction for predicting protein subcellular localizations from bio-images[J]. Front. Comput. Sci., 2017, 11(2): 243-252.
[12] Fengying XIE,Yefen WU,Yang LI,Zhiguo JIANG,Rusong MENG. Adaptive segmentation based on multi-classification model for dermoscopy images[J]. Front. Comput. Sci., 2015, 9(5): 720-728.
[13] Djamal ZIANI. Feature selection on probabilistic symbolic objects[J]. Front. Comput. Sci., 2014, 8(6): 933-947.
[14] Hongpeng YIN,Jinxing LI,Yi CHAI,Simon X. YANG. A survey on distributed compressed sensing: theory and applications[J]. Front. Comput. Sci., 2014, 8(6): 893-904.
[15] Yin LU,Fuxiang WANG,Xiaoyan LUO,Feng LIU. Novel infrared and visible image fusion method based on independent component analysis[J]. Front. Comput. Sci., 2014, 8(2): 243-254.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed