Please wait a minute...
Frontiers of Computer Science

ISSN 2095-2228

ISSN 2095-2236(Online)

CN 10-1014/TP

Postal Subscription Code 80-970

2018 Impact Factor: 1.129

Front. Comput. Sci.    2020, Vol. 14 Issue (4) : 144303    https://doi.org/10.1007/s11704-019-8308-9
RESEARCH ARTICLE
Bayesian compressive principal component analysis
Di MA1,2, Songcan CHEN1,2()
1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China
2. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing 211106, China
 Download: PDF(951 KB)  
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Principal component analysis (PCA) is a widely used method for multivariate data analysis that projects the original high-dimensional data onto a low-dimensional subspace with maximum variance. However, in practice, we would be more likely to obtain a few compressed sensing (CS) measurements than the complete high-dimensional data due to the high cost of data acquisition and storage. In this paper, we propose a novel Bayesian algorithm for learning the solutions of PCA for the original data just from these CS measurements. To this end, we utilize a generative latent variable model incorporated with a structure prior to model both sparsity of the original data and effective dimensionality of the latent space. The proposed algorithm enjoys two important advantages: 1) The effective dimensionality of the latent space can be determined automatically with no need to be pre-specified; 2) The sparsity modeling makes us unnecessary to employ multiple measurement matrices to maintain the original data space but a single one, thus being storage efficient. Experimental results on synthetic and realworld datasets show that the proposed algorithm can accurately learn the solutions of PCA for the original data, which can in turn be applied in reconstruction task with favorable results.

Keywords compressed sensing      principal component analysis      Bayesian learning      sparsity modeling     
Corresponding Author(s): Songcan CHEN   
Just Accepted Date: 18 March 2019   Issue Date: 11 March 2020
 Cite this article:   
Di MA,Songcan CHEN. Bayesian compressive principal component analysis[J]. Front. Comput. Sci., 2020, 14(4): 144303.
 URL:  
https://academic.hep.com.cn/fcs/EN/10.1007/s11704-019-8308-9
https://academic.hep.com.cn/fcs/EN/Y2020/V14/I4/144303
1 I T Jolliffe. Principal Component Analysis. 2nd ed. New York: Springer, 2002
2 C Clausen, H Wechsler. Color image compression using PCA and backpropagation learning. Pattern Recognition, 2000, 33(9): 1555–1560
https://doi.org/10.1016/S0031-3203(99)00126-0
3 Q Du, J E Fowler. Hyperspectral image compression using JPEG2000 and principal component analysis. IEEE Geoscience and Remote Sensing Letters, 2007, 4(2): 201–205
https://doi.org/10.1109/LGRS.2006.888109
4 R Rosipal, M Girolami, L J Trejo, A Cichocki. Kernel PCA for feature extraction and de-noising in nonlinear regression. Neural Computing and Applications, 2001, 10(3): 231–243
https://doi.org/10.1007/s521-001-8051-z
5 L Zhang, W Dong, D Zhang, G Shi. Two-stage image denoising by principal component analysis with local pixel grouping. Pattern Recognition, 2010, 43(4): 1531–1549
https://doi.org/10.1016/j.patcog.2009.09.023
6 G Chen, S Qian. Denoising of hyperspectral imagery using principal component analysis and wavelet shrinkage. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(3): 973–980
https://doi.org/10.1109/TGRS.2010.2075937
7 C E Thomaz, G A Giraldi. A new ranking method for principal components analysis and its application to face image analysis. Image and Vision Computing, 2010, 28(6): 902–913
https://doi.org/10.1016/j.imavis.2009.11.005
8 J Yang, D Zhang, A F Frangi, J Y Yang. Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(1): 131–137
https://doi.org/10.1109/TPAMI.2004.1261097
9 X Luan, B Fang, L Liu, W Yang, J Qian. Extracting sparse error of robust PCA for face recognition in the presence of varying illumination and occlusion. Pattern Recognition, 2014, 47(2): 495–508
https://doi.org/10.1016/j.patcog.2013.06.031
10 L Malagonborja, O Fuentes. Object detection using image reconstruction with PCA. Image and Vision Computing, 2009, 27(1–2): 2–9
https://doi.org/10.1016/j.imavis.2007.03.004
11 D L Donoho. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306
https://doi.org/10.1109/TIT.2006.871582
12 Y C Eldar, G Kutyniok. Compressed Sensing: Theory and Applications. Cambridge: Cambridge University Press, 2012
13 H Qi, S M Hughes. Invariance of principal components under lowdimensional random projection of the data. In: Proceedings of International Conference on Image Processing. 2012, 937–940
https://doi.org/10.1109/ICIP.2012.6467015
14 F P Anaraki, S M Hughes. Memory and computation efficient PCA via very sparse random projections. In: Proceedings of International Conference on Machine Learning. 2014, 1341–1349
15 F P Anaraki. Estimation of the sample covariance matrix from compressive measurements. IET Signal Processing, 2016, 10(9): 1089–1095
https://doi.org/10.1049/iet-spr.2016.0169
16 F P Anaraki, S Becker. Preconditioned data sparsification for big data with applications to PCA and K-means. IEEE Transactions on Information Theory, 2017, 63(5): 2954–2974
17 X Chen, M R Lyu, I King. Toward efficient and accurate covariance matrix estimation on compressed data. In: Proceedings of International Conference on Machine Learning. 2017, 767–776
18 M E Tipping, C M Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 1999, 61(3): 611–622
https://doi.org/10.1111/1467-9868.00196
19 C M Bishop. Bayesian PCA. In: Proceedings of the 11th International Conference on Neural Information Processing Systems. 1999, 382–388
20 A Papoulis, S U Pillai. Probability, Random Variables, and Stochastic Processes. 4th ed. New York: McGraw-Hill Companies, Inc., 2002
21 S Ji, D Dunson, L Carin. Multi-task compressive sensing. IEEE Transactions on Signal Processing, 2009, 57(1): 92–106
https://doi.org/10.1109/TSP.2008.2005866
22 O Stegle, C Lippert, J M Mooij, N D Lawrence, K M Borgwardt. Efficient inference in matrix-variate gaussian models with iid observation noise. In: Proceedings of the 25th Annual Conference on Neural Information Processing Systems. 2012, 630–638
23 C M Bishop. Variational principal components. In: Proceedings of International Conference on Artificial Neural Networks. 1999, 509–514
https://doi.org/10.1049/cp:19991160
24 A Ilin, T Raiko. Practical approaches to principal component analysis in the presence of missing values. Journal of Machine Learning Research, 2010, 11(1): 1957–2000
25 Y L Lecun, L Bottou, Y Bengio, P Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278–2324
https://doi.org/10.1109/5.726791
26 K C Lee, J Ho, D J Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(5): 684–698
https://doi.org/10.1109/TPAMI.2005.92
27 D B Graham, N M Allinson. Characterizing virtual eigensignatures for general purpose face recognition. Face Recognition: From Theory to Applications, 1998, 163(2): 446–456
https://doi.org/10.1007/978-3-642-72201-1_25
28 D Cai, X F He, J W Han. Spectral regression for efficient regularized subspace learning. In: Proceedings of International Conference on Computer Vision. 2007, 1–8
https://doi.org/10.1109/ICCV.2007.4408855
[1] FCS-0003-18308-DM_suppl_1 Download
[1] Kang LI, Fazhi HE, Haiping YU, Xiao CHEN. A parallel and robust object tracking approach synthesizing adaptive Bayesian learning and improved incremental subspace learning[J]. Front. Comput. Sci., 2019, 13(5): 1116-1135.
[2] Jun XIAO, Sidong LIU, Liang HU, Ying WANG. Filtering method of rock points based on BP neural network and principal component analysis[J]. Front. Comput. Sci., 2018, 12(6): 1149-1159.
[3] Zhilin LI, Wenbo XU, Xiaobo ZHANG, Jiaru LIN. A survey on one-bit compressed sensing: theory and applications[J]. Front. Comput. Sci., 2018, 12(2): 217-230.
[4] Feifei ZHANG,Yongbin YU,Qirong MAO,Jianping GOU,Yongzhao ZHAN. Pose-robust feature learning for facial expression recognition[J]. Front. Comput. Sci., 2016, 10(5): 832-844.
[5] Kezhi LI,Shuang CONG. State of the art and prospects of structured sensing matrices in compressed sensing[J]. Front. Comput. Sci., 2015, 9(5): 665-677.
[6] Qiang LI,Yahong HAN,Jianwu DANG. Image decomposing for inpainting using compressed sensing in DCT domain[J]. Front. Comput. Sci., 2014, 8(6): 905-915.
[7] Hongpeng YIN,Jinxing LI,Yi CHAI,Simon X. YANG. A survey on distributed compressed sensing: theory and applications[J]. Front. Comput. Sci., 2014, 8(6): 893-904.
[8] Jianhua JIA, Bingxiang LIU, Licheng JIAO. Soft spectral clustering ensemble applied to image segmentation[J]. Front Comput Sci Chin, 2011, 5(1): 66-78.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed