|
|
Bayesian compressive principal component analysis |
Di MA1,2, Songcan CHEN1,2( ) |
1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China 2. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing 211106, China |
|
|
Abstract Principal component analysis (PCA) is a widely used method for multivariate data analysis that projects the original high-dimensional data onto a low-dimensional subspace with maximum variance. However, in practice, we would be more likely to obtain a few compressed sensing (CS) measurements than the complete high-dimensional data due to the high cost of data acquisition and storage. In this paper, we propose a novel Bayesian algorithm for learning the solutions of PCA for the original data just from these CS measurements. To this end, we utilize a generative latent variable model incorporated with a structure prior to model both sparsity of the original data and effective dimensionality of the latent space. The proposed algorithm enjoys two important advantages: 1) The effective dimensionality of the latent space can be determined automatically with no need to be pre-specified; 2) The sparsity modeling makes us unnecessary to employ multiple measurement matrices to maintain the original data space but a single one, thus being storage efficient. Experimental results on synthetic and realworld datasets show that the proposed algorithm can accurately learn the solutions of PCA for the original data, which can in turn be applied in reconstruction task with favorable results.
|
Keywords
compressed sensing
principal component analysis
Bayesian learning
sparsity modeling
|
Corresponding Author(s):
Songcan CHEN
|
Just Accepted Date: 18 March 2019
Issue Date: 11 March 2020
|
|
1 |
I T Jolliffe. Principal Component Analysis. 2nd ed. New York: Springer, 2002
|
2 |
C Clausen, H Wechsler. Color image compression using PCA and backpropagation learning. Pattern Recognition, 2000, 33(9): 1555–1560
https://doi.org/10.1016/S0031-3203(99)00126-0
|
3 |
Q Du, J E Fowler. Hyperspectral image compression using JPEG2000 and principal component analysis. IEEE Geoscience and Remote Sensing Letters, 2007, 4(2): 201–205
https://doi.org/10.1109/LGRS.2006.888109
|
4 |
R Rosipal, M Girolami, L J Trejo, A Cichocki. Kernel PCA for feature extraction and de-noising in nonlinear regression. Neural Computing and Applications, 2001, 10(3): 231–243
https://doi.org/10.1007/s521-001-8051-z
|
5 |
L Zhang, W Dong, D Zhang, G Shi. Two-stage image denoising by principal component analysis with local pixel grouping. Pattern Recognition, 2010, 43(4): 1531–1549
https://doi.org/10.1016/j.patcog.2009.09.023
|
6 |
G Chen, S Qian. Denoising of hyperspectral imagery using principal component analysis and wavelet shrinkage. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(3): 973–980
https://doi.org/10.1109/TGRS.2010.2075937
|
7 |
C E Thomaz, G A Giraldi. A new ranking method for principal components analysis and its application to face image analysis. Image and Vision Computing, 2010, 28(6): 902–913
https://doi.org/10.1016/j.imavis.2009.11.005
|
8 |
J Yang, D Zhang, A F Frangi, J Y Yang. Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(1): 131–137
https://doi.org/10.1109/TPAMI.2004.1261097
|
9 |
X Luan, B Fang, L Liu, W Yang, J Qian. Extracting sparse error of robust PCA for face recognition in the presence of varying illumination and occlusion. Pattern Recognition, 2014, 47(2): 495–508
https://doi.org/10.1016/j.patcog.2013.06.031
|
10 |
L Malagonborja, O Fuentes. Object detection using image reconstruction with PCA. Image and Vision Computing, 2009, 27(1–2): 2–9
https://doi.org/10.1016/j.imavis.2007.03.004
|
11 |
D L Donoho. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306
https://doi.org/10.1109/TIT.2006.871582
|
12 |
Y C Eldar, G Kutyniok. Compressed Sensing: Theory and Applications. Cambridge: Cambridge University Press, 2012
|
13 |
H Qi, S M Hughes. Invariance of principal components under lowdimensional random projection of the data. In: Proceedings of International Conference on Image Processing. 2012, 937–940
https://doi.org/10.1109/ICIP.2012.6467015
|
14 |
F P Anaraki, S M Hughes. Memory and computation efficient PCA via very sparse random projections. In: Proceedings of International Conference on Machine Learning. 2014, 1341–1349
|
15 |
F P Anaraki. Estimation of the sample covariance matrix from compressive measurements. IET Signal Processing, 2016, 10(9): 1089–1095
https://doi.org/10.1049/iet-spr.2016.0169
|
16 |
F P Anaraki, S Becker. Preconditioned data sparsification for big data with applications to PCA and K-means. IEEE Transactions on Information Theory, 2017, 63(5): 2954–2974
|
17 |
X Chen, M R Lyu, I King. Toward efficient and accurate covariance matrix estimation on compressed data. In: Proceedings of International Conference on Machine Learning. 2017, 767–776
|
18 |
M E Tipping, C M Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 1999, 61(3): 611–622
https://doi.org/10.1111/1467-9868.00196
|
19 |
C M Bishop. Bayesian PCA. In: Proceedings of the 11th International Conference on Neural Information Processing Systems. 1999, 382–388
|
20 |
A Papoulis, S U Pillai. Probability, Random Variables, and Stochastic Processes. 4th ed. New York: McGraw-Hill Companies, Inc., 2002
|
21 |
S Ji, D Dunson, L Carin. Multi-task compressive sensing. IEEE Transactions on Signal Processing, 2009, 57(1): 92–106
https://doi.org/10.1109/TSP.2008.2005866
|
22 |
O Stegle, C Lippert, J M Mooij, N D Lawrence, K M Borgwardt. Efficient inference in matrix-variate gaussian models with iid observation noise. In: Proceedings of the 25th Annual Conference on Neural Information Processing Systems. 2012, 630–638
|
23 |
C M Bishop. Variational principal components. In: Proceedings of International Conference on Artificial Neural Networks. 1999, 509–514
https://doi.org/10.1049/cp:19991160
|
24 |
A Ilin, T Raiko. Practical approaches to principal component analysis in the presence of missing values. Journal of Machine Learning Research, 2010, 11(1): 1957–2000
|
25 |
Y L Lecun, L Bottou, Y Bengio, P Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278–2324
https://doi.org/10.1109/5.726791
|
26 |
K C Lee, J Ho, D J Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(5): 684–698
https://doi.org/10.1109/TPAMI.2005.92
|
27 |
D B Graham, N M Allinson. Characterizing virtual eigensignatures for general purpose face recognition. Face Recognition: From Theory to Applications, 1998, 163(2): 446–456
https://doi.org/10.1007/978-3-642-72201-1_25
|
28 |
D Cai, X F He, J W Han. Spectral regression for efficient regularized subspace learning. In: Proceedings of International Conference on Computer Vision. 2007, 1–8
https://doi.org/10.1109/ICCV.2007.4408855
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|