Please wait a minute...
Shopping Cart Email List Chinese
Advanced Search Fig/Tab
Frontiers of Optoelectronics

ISSN 2095-2759

ISSN 2095-2767(Online)

CN 10-1029/TN

Postal Subscription Code 80-976

Featured Articles  |  Most Downloaded  |  Most Reading
Online Submission Subscribe to Our RSS Content Feed
Front. Optoelectron.    2019, Vol. 12 Issue (4) : 413-421    https://doi.org/10.1007/s12200-019-0862-0
RESEARCH ARTICLE
Robot visual guide with Fourier-Mellin based visual tracking
Chao PENG, Danhua CAO(), Yubin WU, Qun YANG
School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan 430074, China
 Download: PDF(3230 KB)   HTML
 Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Robot vision guide is an important research area in industrial automation, and image-based target pose estimation is one of the most challenging problems. We focus on target pose estimation and present a solution based on the binocular stereo vision in this paper. To improve the robustness and speed of pose estimation, we propose a novel visual tracking algorithm based on Fourier-Mellin transform to extract the target region. We evaluate the proposed tracking algorithm on online tracking benchmark-50 (OTB-50) and the results show that it outperforms other lightweight trackers, especially when the target is rotated or scaled. The final experiment proves that the improved pose estimation approach can achieve a position accuracy of 1.84 mm and a speed of 7 FPS (frames per second). Besides, this approach is robust to the variances of illumination and can work well in the range of 250-700 lux.
Keywords robot visual guide      target pose estimation      stereo vision      visual tracking      Fourier-Mellin transform (FMT)     
Corresponding Author(s): Danhua CAO   
Online First Date: 11 June 2019    Issue Date: 30 December 2019
 Cite this article:   
Chao PENG,Danhua CAO,Yubin WU, et al. Robot visual guide with Fourier-Mellin based visual tracking[J]. Front. Optoelectron., 2019, 12(4): 413-421.
 URL:  
https://academic.hep.com.cn/foe/EN/10.1007/s12200-019-0862-0
https://academic.hep.com.cn/foe/EN/Y2019/V12/I4/413
Fig.1  Robot visual guide algorithm workflow
Fig.2  Flowchart of our tracker, red line (III) represents training stage of last frame, blue line (I, II) represents detect stage of current frame
Fig.3  Robot visual guide experiment system
sequence ours KCF CXT VTD Struck SCM ASLA CSK L1APG attributes
Walking 0.67 0.53 0.17 0.61 0.57 0.71 0.77 0.54 0.75 SV
Walking2 0.62 0.40 0.37 0.33 0.51 0.80 0.37 0.46 0.76 SV
Car2 0.79 0.68 0.87 0.80 0.69 0.92 0.87 0.69 0.91 SV
Car24 0.73 0.43 0.79 0.35 0.14 0.88 0.80 0.38 0.78 SV
Car4 0.76 0.48 0.31 0.36 0.49 0.76 0.75 0.47 0.25 SV
Jogging1 0.66 0.19 0.77 0.15 0.17 0.18 0.18 0.18 0.17 OPR
Singer1 0.67 0.35 0.49 0.49 0.36 0.86 0.79 0.36 0.28 SV OPR
Human6 0.59 0.21 0.15 0.18 0.21 0.35 0.38 0.21 0.25 SV OPR
Tiger2 0.63 0.35 0.36 0.30 0.54 0.09 0.15 0.17 0.24 IPR OPR
Doll 0.74 0.53 0.75 0.65 0.55 0.83 0.83 0.32 0.45 SV IPR OPR
FPS 50 275 15.3 5.7 20.2 0.51 8.5 362 2 –
Tab.1  Comparison of average overlap score
Fig.4  Success plots on OTB-50, the AUC scores are shown in the legend
Fig.5  Snapshot of our tracker. (a) Tiger2; (b) Walking2
Fig.6  Success rate plot
2.0 mm 2.3 mm 2.5 mm average position error
82.7% 99.3% 100% 1.84 mm
Tab.2  Average distance and success rate in three typical threshold
algorithm time expense/ms
before apply tracking after apply tracking
stereo rectify 7.46 7.46
ROI extract 61.21 32.32
stereo matching 14.18 14.18
pose estimation 83.88 83.88
total 166.73 137.84
Tab.3  Time expense
number 1 2 3 4 5 6 7
illumination/lux 98 125 143 186 217 238 248
5 mm success rate 10% 57% 61% 50% 62% 88% 100%
number 8 9 10 11 12 13 14
illumination/lux 276 298 340 432 534 703 823
5 mm success rate 100% 100% 100% 100% 98% 98% 54%
Tab.4  Detail information of the illumination adaptability experiments
Fig.7  Illumination adaptability experiments
1 A Agrawal, Y Sun, J Barnwell, R Raskar. Vision-guided robot system for picking objects by casting shadows. International Journal of Robotics Research, 2010, 29(2-3): 155–173
2 , Taryudi M S Wang. 3D object pose estimation using stereo vision for object manipulation system. In: Proceedings of International Conference on Applied System Innovation. Sapporo: IEEE, 2017, 1532–1535
3 M Dinham, G Fang, J J Zou. Experiments on automatic seam detection for a MIG welding robot. In: Proceedings of International Conference on Artificial Intelligence and Computational Intelligence. Berlin: Springer, 2011, 390–397
4 A Ryberg, M Ericsson, A K Christiansson, K Eriksson, J Nilsson, M Larsson. Stereo vision for path correction in off-line programmed robot welding. In: Proceedings of IEEE International Conference on Industrial Technology. Vina del Mar: IEEE, 2010, 1700–1705
5 M Dinham, G Fang. Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding. Robotics and Computer-integrated Manufacturing, 2013, 29(5): 288–301
6 W H Yun, J Lee, J H Lee, J Kim. Object recognition and pose estimation for modular manipulation system: overview and initial results. In: Proceedings of International Conference on Ubiquitous Robots and Ambient Intelligence. Jeju: IEEE, 2017, 198–201
7 L Dong, X Yu, L Li, J K E Hoe. HOG based multi-stage object detection and pose recognition for service robot. In: Proceedings of 11th International Conference on Control Automation Robotics & Vision, Singapore: IEEE, 2010, 2495–2500
8 H Nam, B Han. Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016, 4293–4302
9 D Held, S Thrun, S Savarese. Learning to track at 100 FPS with deep regression networks. In: Proceedings of European Conference on Computer Vision. Berlin: Springer, 2016, 749–765
10 J F Henriques, C Rui, P Martins, J Batista. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596
11 D Stricker. Tracking with reference images:a real-time and markerless tracking solution for out-door augmented reality applications. In: Proceedings of Conference on Virtual Reality, Archeology, and Cultural Heritage. Glyfada: DBLP, 2001, 77–82
12 Y Wu, J Lim, M H Yang. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834–1848
13 B Yang, J Yan, Z Lei, S Li. Aggregate channel features for multi-view face detection. In: Proceedings of IEEE International Joint Conference on Biometrics. Clearwater: IEEE, 2014, 1–8
14 H Hirschmuller. Accurate and efficient stereo processing by semi-global matching and mutual information. In: Proceedings of IEEE Computer Society Conference on Computer Vision & Pattern Recognition. San Diego: IEEE, 2005, 807–814
15 Z Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330–1334
16 J Li, H Zhao, T Jiang, X Zhou. Development of a 3D high-precise positioning system based on a planar target and two CCD cameras. In: Proceedings of International Conference on Intelligent Robotics and Applications. Berlin: Springer, 2008, 475–484
17 M Magnusson, H Andreasson, A Nüchter, A J Lilienthal. Appearance-based loop detection from 3D laser data using the normal distributions transform. Journal of Field Robotics, 2010, 26(11–12):892–914
18 Y Li, G Liu. Learning a scale-and-rotation correlation filter for robust visual tracking. In: Proceedings of IEEE International Conference on Image Processing. Phoenix: IEEE, 2016, 454–458
19 K Zhang, L Zhang, M H Yang. Fast Compressive Tracking. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 36(10):2002-2015.
20 H Lu, X Jia, M H Yang. Visual tracking via adaptive structural local sparse appearance model. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012, 1822–1829
21 T B Dinh, N Vo, G Medioni. Context tracker: exploring supporters and distracters in unconstrained environments. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs: IEEE, 2011, 1177–1184
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed