Robust visual tracking based on scale invariance and deep learning
Nan REN1,Junping DU1(),Suguo ZHU1,Linghui LI1,Dan FAN1,JangMyung LEE2
1. Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China 2. Department of Electronics Engineering, Pusan National University, Busan 46241, Korea
Visual tracking is a popular research area in computer vision, which is very difficult to actualize because of challenges such as changes in scale and illumination, rotation, fast motion, and occlusion. Consequently, the focus in this research area is to make tracking algorithms adapt to these changes, so as to implement stable and accurate visual tracking. This paper proposes a visual tracking algorithm that integrates the scale invariance of SURF feature with deep learning to enhance the tracking robustness when the size of the object to be tracked changes significantly. Particle filter is used for motion estimation. The confidence of each particle is computed via a deep neural network, and the result of particle filter is verified and corrected by mean shift because of its computational efficiency and insensitivity to external interference. Both qualitative and quantitative evaluations on challenging benchmark sequences demonstrate that the proposed tracking algorithm performs favorably against several state-of-the-art methods throughout the challenging factors in visual tracking, especially for scale variation.
Jia Y M. Robust control with decoupling performance for steering and traction of 4WS vehicles under velocity-varying motion. IEEE Transactions on Control Systems Technology, 2000, 8(3): 554–569 https://doi.org/10.1109/87.845885
2
Jia Y M. Alternative proofs for improved LMI representations for the analysis and the design of continuous-time systems with polytopic type uncertainty: a predictive approach. IEEE Transactions on Automatic Control, 2003, 48(8): 1413–1416 https://doi.org/10.1109/TAC.2003.815033
3
Jia Y M. General solution to diagonal model matching control of multiple-output-delay systems and its applications in adaptive scheme. Progress in Natural Science, 2009, 19(1): 79–90 https://doi.org/10.1016/j.pnsc.2008.05.019
4
Wang N Y, Yeung D Y. Learning a deep compact image representation for visual tracking. In: Proceedings of Advances in Neural Information Processing Systems. 2013, 809–817
5
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P A. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. The Journal ofMachine Learning Research, 2010, 11: 3371–3408
6
Smeulders A W M, Chu D M, Rita C, Simone C, Afshin D, Mubarak S. Visual tracking: an experimental survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(7): 1442–1468 https://doi.org/10.1109/TPAMI.2013.230
7
Ali A, Jalil A, Niu J, Zhao X K, Rathore S, Ahmed J, Iftikhar M A. Visual object tracking—classical and contemporary approaches. Frontiers of Computer Science, 2016, 10(1): 167–188 https://doi.org/10.1007/s11704-015-4246-3
8
Wu Y, Lim J, Yang M H. Online object tracking: a benchmark. In: proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 9(4): 2411–2418 https://doi.org/10.1109/cvpr.2013.312
9
Wu Y, Lim J, Yang M H. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834–1848 https://doi.org/10.1109/TPAMI.2014.2388226
10
Li X, Dick A, Shen C H, Anton V D H, Wang H Z. Incremental learning of 3D-DCT compact representations for robust visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4): 863–881 https://doi.org/10.1109/TPAMI.2012.166
11
Gao J, Ling H B, Hu W M, Xing J L. Transfer learning based visual tracking with Gaussian processes regression. In: Proceedings of the 13th European Conference on Computer Vision. 2014, 188–203 https://doi.org/10.1007/978-3-319-10578-9_13
12
Henriques J F, Caseiro R, Martins P, Batista J. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596 https://doi.org/10.1109/TPAMI.2014.2345390
13
Li X, Shen C H, Dick A, Zhang Z M, Zhuang Y. Online metricweighted linear representations for robust visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(5): 931–950 https://doi.org/10.1109/TPAMI.2015.2469276
14
Zhou Y, Bai X, Liu W Y, Latecki L J. Similarity fusion for visual tracking. International Journal of Computer Vision, 2016, 118(3): 337–363 https://doi.org/10.1007/s11263-015-0879-9
15
Zhong W, Lu H C, Yang M H. Robust object tracking via sparsitybased collaborative model. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1838–1845
16
Hare S, Saffari A, Torr P H S. Struck: structured output tracking with kernels. In: proceedings of IEEE Conference on Computer Vision. 2011, 263–270 https://doi.org/10.1109/iccv.2011.6126251
17
Li X, Dick A, Shen C H, Zhang Z F, Hengel A V D, Wang H Z. Visual tracking with spatio-temporal Dempster-Shafer information fusion. IEEE Transactions on Image Processing, 2013, 22(8): 3028–3040 https://doi.org/10.1109/TIP.2013.2253478
18
Gao C X, Chen F F, Yu J G, Huang R, Sang N. Robust visual tracking using exemplar-based detectors. IEEE Transactions on Circuits and Systems for Video Technology, 2015
19
Li K, He F Z, Chen X. Real-time object tracking via compressive feature selection. Frontiers of Computer Science, 2016, 10(4): 689–701 https://doi.org/10.1007/s11704-016-5106-5
20
Zhang B C, Perina A, Li Z G, Murino V, Liu J Z, Ji R R. Bounding multiple gaussians uncertainty with application to object tracking. International Journal of Computer Vision, 2016, 118(3): 364–379 https://doi.org/10.1007/s11263-016-0880-y
21
Zhu Y Y, Zhang C Q, Zhou D Y, Wang X G, Bai X, Liu W Y. Traffic sign detection and recognition using fully convolutional network guided proposals. Neurocomputing, 2016, 214: 758–766 https://doi.org/10.1016/j.neucom.2016.07.009
22
Li H X, Li Y, Porikli F. DeepTrack: learning discriminative feature representations by convolutional neural networks for visual tracking. IEEE Transactions on Image Processing, 2015, 25(4): 1834–1848 https://doi.org/10.1109/TIP.2015.2510583
23
Hong S H, You T G, Kwak S H, Han B H. Online tracking by learning discriminative saliency map with convolutional neural network. 2015, arXiv:1502.06796v1
24
Wang L, Liu T, Wang G, Chan K L, Yang Q X. Video tracking using learned hierarchical features. IEEE Transactions on Image Processing, 2015, 24(4): 1424–1435 https://doi.org/10.1109/TIP.2015.2403231
25
Ma C, Huang J B, Yang X K, Yang M H. Hierarchical convolutional features for visual tracking. In: proceedings of IEEE International Conference on Computer Vision. 2015, 3074–3082 https://doi.org/10.1109/iccv.2015.352
26
Wang N Y, Li S Y, Gupta A, Yeung D Y. Transferring rich feature hierarchies for robust visual tracking. 2015, arXiv:1501.04587v2
27
Zhang K H, Liu Q S, Wu Y, Yang M H. Robust visual tracking via convolutional networks without training. IEEE Transactions on Image Processing, 2016, 25(4): 1779–1792 https://doi.org/10.1109/tip.2016.2531283
28
Held D, Thrun S, Savarese S. Learning to track at 100 fps with deep regression networks. 2016, arXiv:1604.01802
29
Wang L J, Ouyang W L, Wang X G, Lu H C. STCT: sequentially training convolutional networks for visual tracking. In: proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016 https://doi.org/10.1109/cvpr.2016.153
30
Zhai M Y, Roshtkhari M J, Mori G. Deep Learning of appearance models for online object tracking. 2016, arXiv:1607.02568
31
Arulampalam M S, Maskell S, Gordon N, Clapp T. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing, 2002, 50(2): 174–188 https://doi.org/10.1109/78.978374
32
Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 564–577 https://doi.org/10.1109/TPAMI.2003.1195991
33
Torralba A, Fergus R, Freeman W T. 80 million tiny images: a large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(11): 1958–1970 https://doi.org/10.1109/TPAMI.2008.128
34
Zhang J M, Ma S G, Sclaroff S. MEEM: robust tracking via multiple experts using entropy minimization. In: proceedings of European Conference on Computer Vision. 2014, 188–203 https://doi.org/10.1007/978-3-319-10599-4_13
35
He S F, Yang Q X, Lau R W H, Wang J, Yang M H. Visual tracking via locality sensitive histograms. In: proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 2427–2434 https://doi.org/10.1109/cvpr.2013.314
36
Jia X, Lu H C, Yang M H. Visual tracking via adaptive structural local sparse appearance model. In: proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1822–1829
37
Kwon J, Lee K M. Visual tracking decomposition. In: proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2010, 1269–1276 https://doi.org/10.1109/cvpr.2010.5539821
38
Ross D A, Lim J W, Lin R S, Yang M H. Incremental learning for robust visual tracking. International Journal of Computer Vision, 2008, 77(1): 125–141 https://doi.org/10.1007/s11263-007-0075-7
39
Dinh T B, Vo N, Medioni G. Context tracker: exploring supporters and distracters in unconstrained environments. In: proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2011, 1177–1184 https://doi.org/10.1109/cvpr.2011.5995733
