|
|
Locality-constrained framework for face alignment |
Jie ZHANG1,2, Xiaowei ZHAO3, Meina KAN1, Shiguang SHAN1( ), Xiujuan CHAI1, Xilin CHEN1 |
1. Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China 2. University of Chinese Academy of Sciences, Beijing 100049, China 3. Alibaba Group, Hangzhou 311121, China |
|
|
Abstract Although the conventional active appearance model (AAM) has achieved some success for face alignment, it still suffers from the generalization problem when be applied to unseen subjects and images. To deal with the generalization problem of AAM, we first reformulate the original AAM as sparsity-regularized AAM, which can achieve more compact/better shape and appearance priors by selecting nearest neighbors as the bases of the shape and appearance model. To speed up the fitting procedure, the sparsity in sparsity-regularized AAM is approximated by using the locality (i.e., K-nearest neighbor), and thus inducing the locality-constrained active appearancemodel (LC-AAM). The LC-AAM solves a constrained AAM-like fitting problem with the K-nearest neighbors as the bases of shape and appearance model. To alleviate the adverse influence of inaccurate K-nearest neighbor results, the locality constraint is further embedded in the discriminative fitting method denoted as LC-DFM, which can find better K-nearest neighbor results by employing shape-indexed feature, and can also tolerate some inaccurate neighbors benefited from the regression model rather than the generative model in AAM. Extensive experiments on several datasets demonstrate that our methods outperform the state-of-the-arts in both detection accuracy and generalization ability.
|
Keywords
locality-constrained AAM
locality-constrained DFM
face alignment
sparsity-regularization
|
Corresponding Author(s):
Shiguang SHAN
|
Just Accepted Date: 29 September 2017
Online First Date: 03 December 2018
Issue Date: 29 May 2019
|
|
1 |
LWiskott, J M Fellous, NKuiger, CMalsburg. Face recognition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 775–779
https://doi.org/10.1109/34.598235
|
2 |
XLiu, MKan, WWu, SShan, XChen. Viplfacenet: an open source deep face recognition SDK. Frontiers of Computer Science, 2017, 11(2): 208–218
https://doi.org/10.1007/s11704-016-6076-3
|
3 |
DJiang, YHu, SYan, L Zhang, HZhang, WGao. Efficient 3d reconstruction for face recognition. Pattern Recognition, 2005, 38(6): 787–798
https://doi.org/10.1016/j.patcog.2004.11.004
|
4 |
BFasel, J Luettin. Automatic facial expression analysis: a survey. Pattern Recognition, 2003, 36(1): 259–275
https://doi.org/10.1016/S0031-3203(02)00052-3
|
5 |
FZhang, YYu, QMao, J Gou, YZhan. Pose-robust feature learning for facial expression recognition. Frontiers of Computer Science, 2016, 10(5): 832–844
https://doi.org/10.1007/s11704-015-5323-3
|
6 |
HZheng, XGeng. Facial expression recognition via weighted group sparsity. Frontiers of Computer Science, 2017, 11(2): 266–275
https://doi.org/10.1007/s11704-016-5204-4
|
7 |
TCootes, CTaylor, DCooper, J Graham. Active shape models-their training and application. Computer Vision and Image Understanding, 1995, 61(1): 38–59
https://doi.org/10.1006/cviu.1995.1004
|
8 |
TCootes, G Edwards, CTaylor. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6): 681–685
https://doi.org/10.1109/34.927467
|
9 |
RGross, I Matthews, SBaker. Generic vs. person specific active appearance models. Image and Vision Computing, 2005, 23(12): 1080–1093
https://doi.org/10.1016/j.imavis.2005.07.009
|
10 |
XLiu. Generic face alignment using boosted appearance model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2007, 1–8
https://doi.org/10.1109/CVPR.2007.383265
|
11 |
HWu, XLiu, GDoretto. Face alignment via boosted ranking model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2008, 1–8
|
12 |
JSaragih, RGoecke. A nonlinear discriminative approach to AAM fitting. In: Proceedings of the IEEE International Conference on Computer Vision. 2007, 1–8
https://doi.org/10.1109/ICCV.2007.4409106
|
13 |
XXiong, FTorre. Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 532–539
https://doi.org/10.1109/CVPR.2013.75
|
14 |
AAsthana, S Zafeiriou, SCheng, MPantic. Robust discriminative response map fitting with constrained local models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3444–3451
https://doi.org/10.1109/CVPR.2013.442
|
15 |
DLowe. Distinctive image features from scale-invariant key points. International Journal of Computer Vision, 2004, 60(2): 91–110
https://doi.org/10.1023/B:VISI.0000029664.99615.94
|
16 |
PDollár, P Welinder, PPerona. Cascaded pose regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 1078–1085
https://doi.org/10.1109/CVPR.2010.5540094
|
17 |
GTzimiropoulos. Project-out cascaded regression with an application to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3659–3667
https://doi.org/10.1109/CVPR.2015.7298989
|
18 |
DLee, HPark, CYoo. Face alignment using cascade gaussian process regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 4204–4212
https://doi.org/10.1109/CVPR.2015.7299048
|
19 |
VKazemi, J Sullivan. One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1867–1874
https://doi.org/10.1109/CVPR.2014.241
|
20 |
XCao, YWei, FWen, J Sun. Face alignment by explicit shape regression. International Journal of Computer Vision, 2014, 107(2): 177–190
https://doi.org/10.1007/s11263-013-0667-3
|
21 |
TCootes, CTaylor. A mixture model for representing shape variation. Image and Vision Computing, 1999, 17(8): 567–573
https://doi.org/10.1016/S0262-8856(98)00175-9
|
22 |
LMaaten, E Hendriks. Capturing appearance variation in active appearance models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2010, 34–41
|
23 |
M ETipping, C MBishop. Mixtures of probabilistic principal component analyzers. Neural Computation, 1999, 11(2): 443–482
https://doi.org/10.1162/089976699300016728
|
24 |
PEtyngier, F Segonne, RKeriven. Shape priors using manifold learning techniques. In: Proceedings of the IEEE International Conference on Computer Vision. 2007, 1–8
https://doi.org/10.1109/ICCV.2007.4409040
|
25 |
SZhang, YZhan, MDewan, J Huang, DMetaxas, XZhou. Towards robust and effective shape modeling: sparse shape composition. Medical Image Analysis, 2012, 16(1): 265–277
https://doi.org/10.1016/j.media.2011.08.004
|
26 |
DCiregan, UMeier, JSchmidhuber. Multi-column deep neural networks for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 3642–3649
|
27 |
AKrizhevsky, I Sutskever, GHinton. Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012, 1097–1105
|
28 |
CSzegedy, AToshev, DErhan. Deep neural networks for object detection. In: Proceedings of the Advances in Neural Information Processing Systems Conference. 2013, 2553–2561
|
29 |
RGirshick. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 1440–1448
https://doi.org/10.1109/ICCV.2015.169
|
30 |
JLong, E Shelhamer, TDarrell. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3431–3440
https://doi.org/10.1109/CVPR.2015.7298965
|
31 |
YSun, XWang, XTang. Deep convolutional network cascade for facial point detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3476–3483
https://doi.org/10.1109/CVPR.2013.446
|
32 |
YWu, ZWang, QJi. Facial feature tracking under varying facial expressions and face poses based on restricted boltzmann machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3452–3459
https://doi.org/10.1109/CVPR.2013.443
|
33 |
JZhang, SShan, MKan, X Chen. Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Proceedings of the European Conference on Computer Vision. 2014, 1–16
https://doi.org/10.1007/978-3-319-10605-2_1
|
34 |
ZZhang, PLuo, CLoy, X Tang. Learning and transferring multitask deep representation for face alignment. 2014, arXiv preprint arXiv:1408.3967
|
35 |
GTrigeorgis, PSnape, MNicolaou, E Antonakos, SZafeiriou. Mnemonic descent method: a recurrent process applied for end-to-end face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4177–4187
https://doi.org/10.1109/CVPR.2016.453
|
36 |
AJourabloo, XLiu. Large-pose face alignment via CNN-based dense 3d model fitting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4188–4196
https://doi.org/10.1109/CVPR.2016.454
|
37 |
YYang, ZMa, FNie, X Chang, AHauptmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 2015, 113(2): 113–127
https://doi.org/10.1007/s11263-014-0781-x
|
38 |
NGao, SHuang, SChen. Multi-label active learning by model guided distribution matching. Frontiers of Computer Science, 2016, 10(5): 845–855
https://doi.org/10.1007/s11704-016-5421-x
|
39 |
IMatthews, SBaker. Active appearance models revisited. International Journal of Computer Vision, 2004, 60(2): 135–164
https://doi.org/10.1023/B:VISI.0000029666.37597.d3
|
40 |
XZhao, SShan, XChai, X Chen. Locality-constrained active appearance model. In: Proceedings of the Asian Conference on Computer Vision. 2013, 636–647
https://doi.org/10.1007/978-3-642-37331-2_48
|
41 |
NDalal, BTriggs. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 886–893
https://doi.org/10.1109/CVPR.2005.177
|
42 |
KYu, TZhang, YGong. Nonlinear learning using local coordinate coding. In: Proceedings of the 22nd International Conference on Advances in Neural Information Processing Systems. 2009, 2223–2231
|
43 |
XZhao, XChai, ZNiu, C Heng, SShan. Context constrained facial landmark localization based on discontinuous haar-like feature. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 2011, 673–678
https://doi.org/10.1109/FG.2011.5771329
|
44 |
XZhao, XChai, ZNiu, C Heng, SShan. Context modeling for facial landmark detection based on non-adjacent rectangle (NAR) haar-like feature. Image and Vision Computing, 2012, 30(3): 136–146
https://doi.org/10.1016/j.imavis.2011.12.004
|
45 |
TSim, SBaker, MBsat. The CMU pose, illumination, and expression (PIE) database. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 2002, 46–51
https://doi.org/10.1109/AFGR.2002.1004130
|
46 |
PPhillips, PFlynn, TScruggs, K Bowyer, JChang, KHoffman, J Marques, JMin, WWorek. Overview of the face recognition grand challenge. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 947–954
https://doi.org/10.1109/CVPR.2005.268
|
47 |
PPhillips, H Wechsler, JHuang, PRauss. The feret database and evaluation procedure for face recognition algorithms. Image and Vision Computing, 1998, 16(5): 295–306
https://doi.org/10.1016/S0262-8856(97)00070-X
|
48 |
WGao, BCao, SShan, X Chen, DZhou, XZhang, DZhao. The CASPEAL large-scale chinese face database and baseline evaluations. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 2008, 38(1): 149–161
https://doi.org/10.1109/TSMCA.2007.909557
|
49 |
NKumar, ABerg, PBelhumeur, S Nayar. Attribute and simile classifiers for face verification. In: Proceedings of the IEEE International Conference on Computer Vision. 2009, 365–372
https://doi.org/10.1109/ICCV.2009.5459250
|
50 |
YTian, TKanade, JCohn. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(2): 97–115
https://doi.org/10.1109/34.908962
|
51 |
LGu, TKanade. A generative shape regularization model for robust face alignment. In: Proceedings of European Conference on Computer Vision. 2008, 413–426
https://doi.org/10.1007/978-3-540-88682-2_32
|
52 |
SMilborrow, F Nicolls. Locating facial features with an extended active shape model. In: Proceedings of European Conference on Computer Vision. 2008, 504–513
https://doi.org/10.1007/978-3-540-88693-8_37
|
53 |
JSaragih, SLucey, JCohn. Deformable model fitting by regularized landmark mean-shifts. International Journal of Computer Vision, 2011, 91(2): 200–215
https://doi.org/10.1007/s11263-010-0380-4
|
54 |
MNorouzi, A Punjani, DFleet. Fast search in hamming space with multi-index hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 3108–3115
https://doi.org/10.1109/CVPR.2012.6248043
|
55 |
XLiu, CDeng, BLang, D Tao, XLi. Query-adaptive reciprocal hash tables for nearest neighbor search. IEEE Transactions on Image Processing, 2016, 25(2): 907–919
https://doi.org/10.1109/TIP.2015.2505180
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|