1. Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China 2. University of Chinese Academy of Sciences, Beijing 100049, China 3. Alibaba Group, Hangzhou 311121, China
Although the conventional active appearance model (AAM) has achieved some success for face alignment, it still suffers from the generalization problem when be applied to unseen subjects and images. To deal with the generalization problem of AAM, we first reformulate the original AAM as sparsity-regularized AAM, which can achieve more compact/better shape and appearance priors by selecting nearest neighbors as the bases of the shape and appearance model. To speed up the fitting procedure, the sparsity in sparsity-regularized AAM is approximated by using the locality (i.e., K-nearest neighbor), and thus inducing the locality-constrained active appearancemodel (LC-AAM). The LC-AAM solves a constrained AAM-like fitting problem with the K-nearest neighbors as the bases of shape and appearance model. To alleviate the adverse influence of inaccurate K-nearest neighbor results, the locality constraint is further embedded in the discriminative fitting method denoted as LC-DFM, which can find better K-nearest neighbor results by employing shape-indexed feature, and can also tolerate some inaccurate neighbors benefited from the regression model rather than the generative model in AAM. Extensive experiments on several datasets demonstrate that our methods outperform the state-of-the-arts in both detection accuracy and generalization ability.
LWiskott, J M Fellous, NKuiger, CMalsburg. Face recognition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 775–779 https://doi.org/10.1109/34.598235
2
XLiu, MKan, WWu, SShan, XChen. Viplfacenet: an open source deep face recognition SDK. Frontiers of Computer Science, 2017, 11(2): 208–218 https://doi.org/10.1007/s11704-016-6076-3
3
DJiang, YHu, SYan, L Zhang, HZhang, WGao. Efficient 3d reconstruction for face recognition. Pattern Recognition, 2005, 38(6): 787–798 https://doi.org/10.1016/j.patcog.2004.11.004
HZheng, XGeng. Facial expression recognition via weighted group sparsity. Frontiers of Computer Science, 2017, 11(2): 266–275 https://doi.org/10.1007/s11704-016-5204-4
7
TCootes, CTaylor, DCooper, J Graham. Active shape models-their training and application. Computer Vision and Image Understanding, 1995, 61(1): 38–59 https://doi.org/10.1006/cviu.1995.1004
8
TCootes, G Edwards, CTaylor. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6): 681–685 https://doi.org/10.1109/34.927467
9
RGross, I Matthews, SBaker. Generic vs. person specific active appearance models. Image and Vision Computing, 2005, 23(12): 1080–1093 https://doi.org/10.1016/j.imavis.2005.07.009
10
XLiu. Generic face alignment using boosted appearance model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2007, 1–8 https://doi.org/10.1109/CVPR.2007.383265
11
HWu, XLiu, GDoretto. Face alignment via boosted ranking model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2008, 1–8
12
JSaragih, RGoecke. A nonlinear discriminative approach to AAM fitting. In: Proceedings of the IEEE International Conference on Computer Vision. 2007, 1–8 https://doi.org/10.1109/ICCV.2007.4409106
13
XXiong, FTorre. Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 532–539 https://doi.org/10.1109/CVPR.2013.75
14
AAsthana, S Zafeiriou, SCheng, MPantic. Robust discriminative response map fitting with constrained local models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3444–3451 https://doi.org/10.1109/CVPR.2013.442
PDollár, P Welinder, PPerona. Cascaded pose regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 1078–1085 https://doi.org/10.1109/CVPR.2010.5540094
17
GTzimiropoulos. Project-out cascaded regression with an application to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3659–3667 https://doi.org/10.1109/CVPR.2015.7298989
18
DLee, HPark, CYoo. Face alignment using cascade gaussian process regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 4204–4212 https://doi.org/10.1109/CVPR.2015.7299048
19
VKazemi, J Sullivan. One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1867–1874 https://doi.org/10.1109/CVPR.2014.241
20
XCao, YWei, FWen, J Sun. Face alignment by explicit shape regression. International Journal of Computer Vision, 2014, 107(2): 177–190 https://doi.org/10.1007/s11263-013-0667-3
LMaaten, E Hendriks. Capturing appearance variation in active appearance models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2010, 34–41
PEtyngier, F Segonne, RKeriven. Shape priors using manifold learning techniques. In: Proceedings of the IEEE International Conference on Computer Vision. 2007, 1–8 https://doi.org/10.1109/ICCV.2007.4409040
25
SZhang, YZhan, MDewan, J Huang, DMetaxas, XZhou. Towards robust and effective shape modeling: sparse shape composition. Medical Image Analysis, 2012, 16(1): 265–277 https://doi.org/10.1016/j.media.2011.08.004
26
DCiregan, UMeier, JSchmidhuber. Multi-column deep neural networks for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 3642–3649
27
AKrizhevsky, I Sutskever, GHinton. Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012, 1097–1105
28
CSzegedy, AToshev, DErhan. Deep neural networks for object detection. In: Proceedings of the Advances in Neural Information Processing Systems Conference. 2013, 2553–2561
29
RGirshick. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 1440–1448 https://doi.org/10.1109/ICCV.2015.169
30
JLong, E Shelhamer, TDarrell. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3431–3440 https://doi.org/10.1109/CVPR.2015.7298965
31
YSun, XWang, XTang. Deep convolutional network cascade for facial point detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3476–3483 https://doi.org/10.1109/CVPR.2013.446
32
YWu, ZWang, QJi. Facial feature tracking under varying facial expressions and face poses based on restricted boltzmann machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3452–3459 https://doi.org/10.1109/CVPR.2013.443
33
JZhang, SShan, MKan, X Chen. Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Proceedings of the European Conference on Computer Vision. 2014, 1–16 https://doi.org/10.1007/978-3-319-10605-2_1
34
ZZhang, PLuo, CLoy, X Tang. Learning and transferring multitask deep representation for face alignment. 2014, arXiv preprint arXiv:1408.3967
35
GTrigeorgis, PSnape, MNicolaou, E Antonakos, SZafeiriou. Mnemonic descent method: a recurrent process applied for end-to-end face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4177–4187 https://doi.org/10.1109/CVPR.2016.453
36
AJourabloo, XLiu. Large-pose face alignment via CNN-based dense 3d model fitting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4188–4196 https://doi.org/10.1109/CVPR.2016.454
37
YYang, ZMa, FNie, X Chang, AHauptmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 2015, 113(2): 113–127 https://doi.org/10.1007/s11263-014-0781-x
38
NGao, SHuang, SChen. Multi-label active learning by model guided distribution matching. Frontiers of Computer Science, 2016, 10(5): 845–855 https://doi.org/10.1007/s11704-016-5421-x
XZhao, SShan, XChai, X Chen. Locality-constrained active appearance model. In: Proceedings of the Asian Conference on Computer Vision. 2013, 636–647 https://doi.org/10.1007/978-3-642-37331-2_48
41
NDalal, BTriggs. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 886–893 https://doi.org/10.1109/CVPR.2005.177
42
KYu, TZhang, YGong. Nonlinear learning using local coordinate coding. In: Proceedings of the 22nd International Conference on Advances in Neural Information Processing Systems. 2009, 2223–2231
43
XZhao, XChai, ZNiu, C Heng, SShan. Context constrained facial landmark localization based on discontinuous haar-like feature. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 2011, 673–678 https://doi.org/10.1109/FG.2011.5771329
44
XZhao, XChai, ZNiu, C Heng, SShan. Context modeling for facial landmark detection based on non-adjacent rectangle (NAR) haar-like feature. Image and Vision Computing, 2012, 30(3): 136–146 https://doi.org/10.1016/j.imavis.2011.12.004
45
TSim, SBaker, MBsat. The CMU pose, illumination, and expression (PIE) database. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 2002, 46–51 https://doi.org/10.1109/AFGR.2002.1004130
46
PPhillips, PFlynn, TScruggs, K Bowyer, JChang, KHoffman, J Marques, JMin, WWorek. Overview of the face recognition grand challenge. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 947–954 https://doi.org/10.1109/CVPR.2005.268
47
PPhillips, H Wechsler, JHuang, PRauss. The feret database and evaluation procedure for face recognition algorithms. Image and Vision Computing, 1998, 16(5): 295–306 https://doi.org/10.1016/S0262-8856(97)00070-X
48
WGao, BCao, SShan, X Chen, DZhou, XZhang, DZhao. The CASPEAL large-scale chinese face database and baseline evaluations. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 2008, 38(1): 149–161 https://doi.org/10.1109/TSMCA.2007.909557
49
NKumar, ABerg, PBelhumeur, S Nayar. Attribute and simile classifiers for face verification. In: Proceedings of the IEEE International Conference on Computer Vision. 2009, 365–372 https://doi.org/10.1109/ICCV.2009.5459250
50
YTian, TKanade, JCohn. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(2): 97–115 https://doi.org/10.1109/34.908962
51
LGu, TKanade. A generative shape regularization model for robust face alignment. In: Proceedings of European Conference on Computer Vision. 2008, 413–426 https://doi.org/10.1007/978-3-540-88682-2_32
52
SMilborrow, F Nicolls. Locating facial features with an extended active shape model. In: Proceedings of European Conference on Computer Vision. 2008, 504–513 https://doi.org/10.1007/978-3-540-88693-8_37
53
JSaragih, SLucey, JCohn. Deformable model fitting by regularized landmark mean-shifts. International Journal of Computer Vision, 2011, 91(2): 200–215 https://doi.org/10.1007/s11263-010-0380-4
54
MNorouzi, A Punjani, DFleet. Fast search in hamming space with multi-index hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 3108–3115 https://doi.org/10.1109/CVPR.2012.6248043
55
XLiu, CDeng, BLang, D Tao, XLi. Query-adaptive reciprocal hash tables for nearest neighbor search. IEEE Transactions on Image Processing, 2016, 25(2): 907–919 https://doi.org/10.1109/TIP.2015.2505180
