Real-time manifold regularized context-aware correlation tracking

doi:10.1007/s11704-018-8104-y

Front. Comput. Sci.

2020, Vol. 14

Issue (2) : 334-348 https://doi.org/10.1007/s11704-018-8104-y

RESEARCH ARTICLE

Real-time manifold regularized context-aware correlation tracking

Jiaqing FAN¹, Huihui SONG¹, Kaihua ZHANG¹(

), Qingshan LIU¹, Fei YAN¹, Wei LIAN²

¹. Jiangsu Key Laboratory of Big Data Analysis Technology (B-DAT), Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science & Technology, Nanjing 211800, China
². Department of Computer Science, Changzhi University, Changzhi 046011, China

Download: PDF(2323 KB)
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Despite the demonstrated success of numerous correlation filter (CF) based tracking approaches, their assumption of circulant structure of samples introduces significant redundancy to learn an effective classifier. In this paper, we develop a fast manifold regularized context-aware correlation tracking algorithm that mines the local manifold structure information of different types of samples. First, different from the traditional CF based tracking that only uses one base sample, we employ a set of contextual samples near to the base sample, and impose a manifold structure assumption on them. Afterwards, to take into account the manifold structure among these samples, we introduce a linear graph Laplacian regularized term into the objective of CF learning. Fortunately, the optimization can be efficiently solved in a closed form with fast Fourier transforms (FFTs), which contributes to a highly efficient implementation. Extensive evaluations on the OTB100 and VOT2016 datasets demonstrate that the proposed tracker performs favorably against several state-of-the-art algorithms in terms of accuracy and robustness. Especially, our tracker is able to run in real-time with 28 fps on a single CPU.

Keywords visual tracking manifold regularization correlation filter graph Laplacian

Corresponding Author(s): Kaihua ZHANG

Just Accepted Date: 27 July 2018 Online First Date: 17 September 2019 Issue Date: 16 October 2019

Cite this article:

Jiaqing FAN,Huihui SONG,Kaihua ZHANG, et al. Real-time manifold regularized context-aware correlation tracking[J]. Front. Comput. Sci., 2020, 14(2): 334-348.

URL:

https://academic.hep.com.cn/fcs/EN/10.1007/s11704-018-8104-y
https://academic.hep.com.cn/fcs/EN/Y2020/V14/I2/334

1	X Li, W M Hu, C H Shen, Z F Zhang, A Dick, A V D Hengel. A survey of appearance models in visual object tracking. ACM Transactions on Intelligent Systems and Technology, 2013, 4(4): 1–13 https://doi.org/10.1145/2508037.2508039
2	H J Wang, H J Ge. Visual tracking using discriminative representation with l2 regularization. Frontiers of Computer Science, 2018, 12(1): 1–13 https://doi.org/10.1007/s11704-017-6434-9
3	A Ali, A Jalil, J W Niu, X K Zhao, S Rathore, J Ahmed, M A Iftikhar. Visual object tracking−classical and contemporary approaches. Frontiers of Computer Science, 2016, 10(1): 167–188 https://doi.org/10.1007/s11704-015-4246-3
4	K H Zhang, Q S Liu, J Ynag, M H Yang. Visual tracking via boolean map representations. Pattern Recognition, 2018, 81: 147–160 https://doi.org/10.1016/j.patcog.2018.03.029
5	K H Zhang, X J Li, H H Song, Q S Liu, W Lian. Visual tracking using spatio-temporally nonlocally regularized correlation filter. Pattern Recognition, 2018, 83: 185–195 https://doi.org/10.1016/j.patcog.2018.05.017
6	D S Bolme, J R Beveridge, B A Draper, Y M Lui. Visual object tracking using adaptive correlation filters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 2544–2550 https://doi.org/10.1109/CVPR.2010.5539960
7	J F Henriques, R Caseiro, P Martins, J Batista. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596 https://doi.org/10.1109/TPAMI.2014.2345390
8	K H Zhang, L Zhang, Q S Liu, D Zhang, M H Yang. Fast visual tracking via dense spatio-temporal context learning. In: Proceedings of European Conference on Computer Vision. 2014, 127–141 https://doi.org/10.1007/978-3-319-10602-1_9
9	L Bertinetto, J Valmadre, S Golodetz, O Miksik, P H S Torr. Staple: complementary learners for real-time tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 1401–1409 https://doi.org/10.1109/CVPR.2016.156
10	K H Zhang, Q S Liu, Y Wu, M H Yang. Robust visual tracking via convolutional networks without training. IEEE Transactions on Image Processing, 2016, 25(4): 1779–1792
11	C Ma, Y Xu, B B Ni, X K Yang. When correlation filters meet convolutional neural networks for visual tracking. IEEE Signal Processing Letters, 2016, 23(10): 1454–1458 https://doi.org/10.1109/LSP.2016.2601691
12	M Danelljan, G Häger, F Khan, M Felsberg. Accurate scale estimation for robust visual tracking. In: Proceedings of the British Machine Vision Conference. 2014 https://doi.org/10.5244/C.28.65
13	M Kristan, A Leonardis, J Matas. The visual object tracking VOT2017 challenge results. In: Proceedings of the IEEE International Conference on Computer Vision Workshop. 2017, 1949–1972
14	M Mueller, N Smith, B Ghanem. Context-aware correlation filter tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 1396–1404 https://doi.org/10.1109/CVPR.2017.152
15	K H Galoogahi, A Fagg, S Lucey. Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 1135–1143 https://doi.org/10.1109/ICCV.2017.129
16	Y Yan, F Nie, W Li, C Q Gao, Y Yang, D Xu. Image classification by cross-media active learning with privileged information. IEEE Transactions on Multimedia, 2016, 18(12): 2494–2502 https://doi.org/10.1109/TMM.2016.2602938
17	Y Yang, Z G Ma, F P Nie, X J Chang, A G Hauptmann. Multi-class active learning by uncertainty sampling with diversity maximization. International Journal of Computer Vision, 2015, 113(2): 113–127 https://doi.org/10.1007/s11263-014-0781-x
18	Y Yang, F P Nie, D Xu, J B Luo, Y T Zhuang, Y H Pan. A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(4): 723–742 https://doi.org/10.1109/TPAMI.2011.170
19	Y Wu, J W Lim, M H Yang. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834–1848 https://doi.org/10.1109/TPAMI.2014.2388226
20	M Danelljan, K F Shahbaz, M Felsberg, V W Joost. Adaptive color attributes for real-time visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1090–1097 https://doi.org/10.1109/CVPR.2014.143
21	C Ma, J B Huang, X K Yang, M H Yang. Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 3074–3082 https://doi.org/10.1109/ICCV.2015.352
22	M Danelljan, G Hager, F S Khan, M Felsberg. Convolutional features for correlation filter based visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2015, 58–66 https://doi.org/10.1109/ICCVW.2015.84
23	M Danelljan, A Robinson, F S Khan, M Felsberg. Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Proceedings of European Conference on Computer Vision. 2016, 472–488 https://doi.org/10.1007/978-3-319-46454-1_29
24	M Danelljan, G Bhat, F S Khan, M Felsberg. ECO: rfficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 21–26 https://doi.org/10.1109/CVPR.2017.733
25	S Liu, T Z Zhang, X C Cao, C S Xu. Structural correlation filter for robust visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 4312–4320 https://doi.org/10.1109/CVPR.2016.467
26	A Lukezic, T Vojír, L C Zajc, J Matas, M Kristan. Discriminative correlation filter with channel and spatial reliability. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 6309–6318 https://doi.org/10.1109/CVPR.2017.515
27	M Danelljan, G Hager, K F Shahbaz, M Felsberg. Learning spatially regularized correlation filters for visual tracking. In: Proceedings of European Conference on Computer Vision. 2015, 4310–4318 https://doi.org/10.1109/ICCV.2015.490
28	M Belkin, P Niyogi, V Sindhwani. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 2006, 7(Nov): 2399–2434
29	X J Chang, Y Yang. Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2294–2305 https://doi.org/10.1109/TNNLS.2016.2582746
30	S Yu, Y Yang, A Hauptmann. Harry potter’s marauder’s map: localizing and tracking multiple persons-of-interest by nonnegative discretization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3714–3720 https://doi.org/10.1109/CVPR.2013.476
31	Y C Bai, M Tang. Robust tracking via weakly supervised ranking SVM. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 1854–1861
32	H W Hu, B Ma, J B Shen, L Shao. Manifold regularized correlation object tracking. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(5): 1786–1795 https://doi.org/10.1109/TNNLS.2017.2688448
33	B Zhuang, H C Lu, Z Y Xiao, D Wang. Visual tracking via discriminative sparse similarity map. IEEE Transactions on Image Processing, 2014, 23(4): 1872–1881 https://doi.org/10.1109/TIP.2014.2308414
34	M Belkin, P Niyogi. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Processing Systems, 2001, 585–591
35	C Ma, X K Yang, C Y Zhang, M H Yang. Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5388–5396 https://doi.org/10.1109/CVPR.2015.7299177
36	J M Zhang, S G Ma, S Sclaroff. MEEM: robust tracking via multiple experts using entropy minimization. In: Proceedings of European Conference on Computer Vision. 2014, 188–203 https://doi.org/10.1007/978-3-319-10599-4_13
37	S Hare, S Golodetz, A Saffari, V Vineet, M M Cheng, S L Hicks, P H S Torr. Struck: structured output tracking with kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(10): 2096–2109 https://doi.org/10.1109/TPAMI.2015.2509974
38	Y Wu, J W Lim, M H Yang. Online object tracking: a benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 2411–2418 https://doi.org/10.1109/CVPR.2013.312
39	Y B Song, C Ma, L J Gong, J W Zhang, R W H Lau, M H Yang. Crest: convolutional residual learning for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2574–2583 https://doi.org/10.1109/ICCV.2017.279
40	G Zhu, F Porikli, H D Li. Beyond local search: tracking objects everywhere with instance-specific proposals. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 943–951 https://doi.org/10.1109/CVPR.2016.108

[1]

Article highlights

Download

[1]	Yunyun WANG, Jiao HAN, Yating SHEN, Hui XUE. Pointwise manifold regularization for semi-supervised learning[J]. Front. Comput. Sci., 2021, 15(1): 151303-.
[2]	Zhenyang SU, Jing LI, Jun CHANG, Bo DU, Yafu XIAO. Real-time visual tracking using complementary kernel support correlation filters[J]. Front. Comput. Sci., 2020, 14(2): 417-429.
[3]	Haijun WANG, Hongjuan GE. Visual tracking using discriminative representation with ℓ₂ regularization[J]. Front. Comput. Sci., 2019, 13(1): 199-211.
[4]	Jun ZHANG, Bineng ZHONG, Pengfei WANG, Cheng WANG, Jixiang DU. Robust feature learning for online discriminative tracking without large-scale pre-training[J]. Front. Comput. Sci., 2018, 12(6): 1160-1172.
[5]	Nan REN,Junping DU,Suguo ZHU,Linghui LI,Dan FAN,JangMyung LEE. Robust visual tracking based on scale invariance and deep learning[J]. Front. Comput. Sci., 2017, 11(2): 230-242.

Viewed

Full text

Abstract

Cited

Shared

Discussed