Adaptive multi-filter tracker based on efficient convolution operator

Li Guoyou; Zhang Fengxv; Ji Zhian

doi:10.12086/oee.2020.190510

Article navigation > Opto-Electronic Engineering > 2020 Vol. 47 > No. 7 > 190510

Next Article Previous Article

Li G Y, Zhang F X, Ji Z A. Adaptive multi-filter tracker based on efficient convolution operator[J]. Opto-Electron Eng, 2020, 47(7): 190510. doi: 10.12086/oee.2020.190510

Citation:

Li G Y, Zhang F X, Ji Z A. Adaptive multi-filter tracker based on efficient convolution operator[J]. Opto-Electron Eng, 2020, 47(7): 190510. doi: 10.12086/oee.2020.190510

Adaptive multi-filter tracker based on efficient convolution operator

Key Laboratory of Industrial Computer Control Engineering, Yanshan University, Qinhuangdao, Hebei 066004, China

Fund Project: Supported by Youth Fund for Science and Technology Research in Colleges (2011139), and Universities of Hebei Province and Natural Science Foundation of Hebei Province (F2012203111)

More Information

Corresponding author: Zhang Fengxv, E-mail:1097089954@qq.com

Received Date 31 August 2019

Revised Date 27 December 2019

Published Date 01 July 2020

Abstract

Abstract

With the problem of difficulty that a single filter to adapt to various complex changes in the tracking process, an adaptive multi-filter target tracking algorithm based on the efficient convolution operators for tracking is proposed. Spatial-temporal regularized filter, the consistency check filter and the correlation filter in the efficient convolution operator tracker, convolve with target features respectively, which obtains three detection scores. The training method of spatial-temporal regularized filter is to introduce temporal regularization into loss function. The consistency check filter is a filter that uses current filter to track the target of previous several frames and updates only when the error of forward and backward position is less than the threshold. Target position is estimated by the best filter detection score with the peak-to-side ratio is maximum. The improved algorithm is tested with the OTB-2015 dataset and UAV123 dataset. The experimental results show that the proposed algorithm can better adapt to the complex environment in tracking process, which has high precision and robustness.
- target tracking /
- filter /
- spatial-temporal regularized /
- consistency check /
- efficient convolution operators

FullText(HTML)

References

[1]	卢湖川, 李佩霞, 王栋.目标跟踪算法综述[J].模式识别与人工智能, 2018, 31(1): 61-76. Google Scholar Lu H C, Li P X, Wang D. Visual object tracking: a survey[J]. Pattern Recognition and Artificial Intelligence, 2018, 31(1): 61-76. Google Scholar
[2]	樊香所, 徐智勇, 张建林.改进粒子滤波的弱小目标跟踪[J].光电工程, 2018, 45(8): 170569. doi: 10.12086/oee.2018.170569 CrossRef Google Scholar Fan X S, Xu Z Y, Zhang J L. Dim small target tracking based on improved particle filter[J]. Opto-Electronic Engineering, 2018, 45(8): 170569. doi: 10.12086/oee.2018.170569 CrossRef Google Scholar
[3]	李双双, 赵高鹏, 王建宇.基于特征融合和尺度自适应的干扰感知目标跟踪[J].光学学报, 2017, 37(5): 0515005. Google Scholar Li S S, Zhao G P, Wang J Y. Distractor-aware object tracking based on multi-feature fusion and scale-adaption[J]. Acta Optica Sinica, 2017, 37(5): 0515005. Google Scholar
[4]	Ross D A, Lim J, Lin R S, et al. Incremental learning for robust visual tracking[J]. International Journal of Computer Vision, 2008, 77(1-3): 125-141. doi: 10.1007/s11263-007-0075-7 CrossRef Google Scholar
[5]	Babenko B, Yang M H, Belongie S. Robust object tracking with online multiple instance learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8): 1619-1632. doi: 10.1109/TPAMI.2010.226 CrossRef Google Scholar
[6]	朱文青, 刘艳, 卞乐, 等.基于生成式模型的目标跟踪方法综述[J].微处理机, 2017, 38(1): 41-47. doi: 10.3969/j.issn.1002-2279.2017.01.011 CrossRef Google Scholar Zhu W Q, Liu Y, Bian L, et al. Survey on object tracking method base on generative model[J]. Microprocessors, 2017, 38(1): 41-47. doi: 10.3969/j.issn.1002-2279.2017.01.011 CrossRef Google Scholar
[7]	葛宝义, 左宪章, 胡永江.视觉目标跟踪方法研究综述[J].中国图象图形学报, 2018, 23(8): 1091-1107. Google Scholar Ge B Y, Zuo X Z, Hu Y J. Review of visual object tracking technology[J]. Journal of Image and Graphics, 2018, 23(8): 1091-1107. Google Scholar
[8]	Bolme D S, Beveridge J R, Draper B A, et al. Visual object tracking using adaptive correlation filters[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010. Google Scholar
[9]	Henriques J F, Caseiro R, Martins P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//European Conference on Computer Vision, Berlin, Heidelberg, 2012: 702-715. Google Scholar
[10]	Henriques J F, Caseiro R, Martins P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596. doi: 10.1109/TPAMI.2014.2345390 CrossRef Google Scholar
[11]	Danelljan M, Häger G, Khan F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015: 4310-4318. Google Scholar
[12]	Danelljan M, Robinson A, Khan F S, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Computer Vision-ECCV 2016, Cham, 2016: 472-488. Google Scholar
[13]	Lukežic A, Vojír T, Zajc L C, et al. Discriminative correlation filter with channel and spatial reliability[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017: 4847-4856. Google Scholar
[14]	Danelljan M, Bhat G, Khan F S, et al. ECO: Efficient convolution operators for tracking[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017: 6638-6646. Google Scholar
[15]	Li F, Tian C, Zuo W M, et al. Learning spatial-temporal regularized correlation filters for visual tracking[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 4904-4913. Google Scholar
[16]	胡硕, 赵银妹, 孙翔.基于卷积神经网络的目标跟踪算法综述[J].高技术通讯, 2018, 28(3): 207-213. doi: 10.3772/j.issn.1002-0470.2018.03.003 CrossRef Google Scholar Hu S, Shao Y M, Sun X. Review of object tracking based on convolutional neural networks[J]. High Technology Letters, 2018, 28(3): 207-213. doi: 10.3772/j.issn.1002-0470.2018.03.003 CrossRef Google Scholar
[17]	孙航, 李晶, 杜博, 等.基于多阶段学习的相关滤波目标跟踪[J].电子学报, 2017, 45(10): 2337-2342. doi: 10.3969/j.issn.0372-2112.2017.10.004 CrossRef Google Scholar Sun H, Li J, Du B, et al. Correlation filtering target tracking based on online multi-lifespan learning[J]. Acta Electronica Sinica, 2017, 45(10): 2337-2342. doi: 10.3969/j.issn.0372-2112.2017.10.004 CrossRef Google Scholar
[18]	Wu Y, Lim J, Yang M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848. doi: 10.1109/TPAMI.2014.2388226 CrossRef Google Scholar
[19]	Mueller M, Smith N, Ghanem B. A benchmark and simulator for UAV tracking[C]//Computer Vision-ECCV 2016, Cham, 2016: 445-461. Google Scholar
[20]	Danelljan M, Häger G, Khan F S, et al. Discriminative scale space tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(8): 1561-1575. doi: 10.1109/TPAMI.2016.2609928 CrossRef Google Scholar
[21]	Ma C, Huang J B, Yang X K, et al. Hierarchical convolutional features for visual tracking[C]//2015 IEEE international Conference on Computer Vision (ICCV), Santiago, Chile, 2015: 3074-3082. Google Scholar
[22]	Zhang J M, Ma S G, Sclaroff S. MEEM: robust tracking via multiple experts using entropy minimization[C]//2014 European Conference on Computer Vision, Berlin, 2014: 188-203. Google Scholar
[23]	Bertinetto L, Valmadre J, Golodetz S, et al. Staple: Complementary learners for real-time tracking[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016: 1401-1409. Google Scholar
[24]	Ma C, Yang X K, Zhang C Y, et al. Long-term correlation tracking[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015: 5388-5396. Google Scholar
[25]	Li Y, Zhu J K, Hoi S C H, et al. Robust estimation of similarity transformation for visual object tracking[Z]. arXiv: 1712.05231[cs.CV], 2017. Google Scholar

Overview

Overview

Overview: In the actual target tracking process, the shape and posture of the target are different, and the environment in tracking is complex and changeable. It is difficult for a single filter to cope with the complex changes of the video sequence and tracking environment. To solve this problem, based on the efficient convolution operators for tracking algorithm, a multi-filter target tracking algorithm which can adapt to more complex environments is proposed. The algorithm trains two more filters of spatial-temporal regularization filter and consistency checking filter than the efficient convolution operators for tracking algorithm. The spatial-temporal regularization filter is obtained by introducing the temporal regularization into the loss function of correlation filtering. Spatial-temporal regularization filter can well adapt to the huge changes in the appearance of the target, so it can adapt to the environment that targets variable. The training method of consistency check filter is: firstly, the current filter is used to locate the target that has been tracked forward, and then the errors between the reverse location coordinate and the forward location coordinate are compared. When the error is less than the threshold, the consistency check filter is updated, but not be updated when the distance error is greater than the threshold. The consistency check filter reduces the noise information introduced in the filter update process, so it can be used in the case of more background clutter and noise. The correlation filter in the efficient convolution operators for tracking algorithm retains the most comprehensive target and background feature information, so it is suitable for relatively stable tracking environment with less interference. Spatial-temporal regularization filter, consistency check filter and correlation filter in efficient convolution operators for tracking are convolved with target features respectively, and three filter detection scores are obtained. The filter detection score obeys the Gaussian distribution. The higher the peak to side ratio of detection score, the higher the target tracking accuracy. The position of the target is estimated by the best filter detection score witch the peak to side ratio is more than the other filter detection score. The improved algorithm is evaluated on the OTB-2015 data set and UAV123 data set. Through qualitative and quantitative analysis, the experimental results show that the improved algorithm can better adapt to the complex changing environment in the tracking process, and its accuracy and success are improved, which is superior to most existing tracking algorithms.