Application of aircraft target tracking based on deep learning

Zhao Chunmei; Chen Zhongbi; Zhang Jianlin

doi:10.12086/oee.2019.180261

Article navigation > Opto-Electronic Engineering > 2019 Vol. 46 > No. 9 > 180261

Next Article Previous Article

Zhao Chunmei, Chen Zhongbi, Zhang Jianlin. Application of aircraft target tracking based on deep learning[J]. Opto-Electronic Engineering, 2019, 46(9): 180261. doi: 10.12086/oee.2019.180261

Citation:

Zhao Chunmei, Chen Zhongbi, Zhang Jianlin. Application of aircraft target tracking based on deep learning[J]. Opto-Electronic Engineering, 2019, 46(9): 180261. doi: 10.12086/oee.2019.180261

Application of aircraft target tracking based on deep learning

1.
Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China
2.
University of Chinese Academy of Sciences, Beijing 100049, China

Fund Project: Supported by Major Special Fund (G158207)

More Information

Corresponding author: Chen Zhongbi, E-mail:ioeyoyo@126.com

Received Date 17 May 2018

Revised Date 15 October 2018

Published Date 30 September 2019

Abstract

Abstract

In this paper, based on muti-domain network (MDNet), fast deep learning for aircraft tracking (FDLAT) algorithm is proposed to track aircraft target. This algorithm uses feature-based transfer learning to make up the inferiority of small sample sets, uses specific data sets to update parameters of convolutional layers and fully connected layers, and use it to distinguish aircraft from background. After building the training model, we put the aircraft video sets into the model and tracked the aircraft using regression model and a simple line on-line update, to increase the speed while ensuring the accuracy. This algorithm achieves robust tracking for aircraft in rotation, similar targets, fuzzy targets, complex environment, scale transformation, target occlusion, morphological transformation and other complex states, and runs at a speed of 20.36 frames with the overlap reached 0.592 in the ILSVRC2015 detection sets of aircraft, basically meets the real-time application requirement of aircraft tracking.
- FDLAT /
- feature-based transfer learning /
- aircraft target /
- robust tracking /
- real-time tracking

FullText(HTML)

References

[1]	Sivanantham S, Paul N N, Iyer R S. Object tracking algorithm implementation for security applications[J]. Far East Journal of Electronics and Communications, 2016, 16(1): 1–13. doi: 10.17654/EC016010001 CrossRef Google Scholar
[2]	Kwak S, Cho M, Laptev I, et al. Unsupervised object discovery and tracking in video collections[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 3173–3181.http://www.oalib.com/paper/4075520 Google Scholar
[3]	罗海波, 许凌云, 惠斌, 等.基于深度学习的目标跟踪方法研究现状与展望[J].红外与激光工程, 2017, 46(5): 502002. Google Scholar Luo H B, Xu L Y, Hui B, et al. Status and prospect of target tracking based on deep learning[J]. Infrared and Laser Engineering, 2017, 46(5): 502002. Google Scholar
[4]	Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 564–577. doi: 10.1109/TPAMI.2003.1195991 CrossRef Google Scholar
[5]	Jia X, Lu H C, Yang M H. Visual tracking via adaptive structural local sparse appearance model[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012: 1822–1829.http://www.researchgate.net/publication/261334218_Visual_tracking_via_adaptive_structural_local_sparse_appearance_model Google Scholar
[6]	Henriques J F, Caseiro R, Martins P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583–596. doi: 10.1109/TPAMI.2014.2345390 CrossRef Google Scholar
[7]	樊香所, 徐智勇, 张建林.改进粒子滤波的弱小目标跟踪[J].光电工程, 2018, 45(8): 170569. doi: 10.12086/oee.2018.170569 CrossRef Google Scholar Fan X S, Xu Z Y, Zhang J L. Dim small target tracking based on improved particle filter[J]. Opto-Electronic Engineering, 2018, 45(8): 170569. doi: 10.12086/oee.2018.170569 CrossRef Google Scholar
[8]	奚玉鼎, 于涌, 丁媛媛, 等.一种快速搜索空中低慢小目标的光电系统[J].光电工程, 2018, 45(4): 170654. doi: 10.12086/oee.2018.170654 CrossRef Google Scholar Xi Y D, Yu Y, Ding Y Y, et al. An optoelectronic system for fast search of low slow small target in the air[J]. Opto-Electronic Engineering, 2018, 45(4): 170654. doi: 10.12086/oee.2018.170654 CrossRef Google Scholar
[9]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems, 2012: 1097–1105.http://www.researchgate.net/publication/267960550_ImageNe Google Scholar
[10]	Chatfield K, Simonyan K, Vedaldi A, et al. Return of the devil in the details: delving deep into convolutional nets[J]. arXiv: 1405.3531[cs.CV], 2014. Google Scholar
[11]	Hyeonseob N, Mooyeol B, Bohyung H. Modeling and Propagating CNNs in a Tree Structure for Visual Tracking[J]. arXiv: 1608.07242v1[cs.CV], 2016: 1–10. Google Scholar
[12]	Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640–651. doi: 10.1109/TPAMI.2016.2572683 CrossRef Google Scholar
[13]	Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of 2014 Conference on Computer Vision and Pattern Recognition, 2014: 580–587. Google Scholar
[14]	Bertinetto L, Valmadre J, Henriques J F, et al. Fully-convolutional Siamese networks for object tracking[C]//Proceedings of 2016 European Conference on Computer Vision, 2016: 850–865.http://link.springer.com/chapter/10.1007/978-3-319-48881-3_56 Google Scholar
[15]	Valmadre J, Bertinetto L, Henriques J F, et al. End-to-end representation learning for Correlation Filter based tracking[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017.http://www.researchgate.net/publication/320971954_End-to-End_Representation_Learning_for_Correlation_Filter_Based_Tracking Google Scholar
[16]	Nam H, Han B. Learning multi-domain convolutional neural networks for visual tracking[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 4293–4302.http://www.oalib.com/paper/4053404 Google Scholar
[17]	Held D, Thrun S, Savarese S. Learning to track at 100 FPS with deep regression networks[C]//Proceedings of the 14th European Conference on Computer Vision, 2016: 745–765.http://link.springer.com/chapter/10.1007/978-3-319-46448-0_45 Google Scholar
[18]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv: 1409.1556[cs.CV], 2014. http://www.oalib.com/paper/4068791 Google Scholar
[19]	Chen K, Tao W B. Once for all: a two-flow convolutional neural network for visual tracking[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 28(12): 3377–3386. doi: 10.1109/TCSVT.2017.2757061 CrossRef Google Scholar
[20]	Leal-Taixé L, Canton-Ferrer C, Schindler C. Learning by tracking: Siamese CNN for robust target association[C]//Proceedings of 2016 Computer Vision and Pattern Recognition Workshops, 2016: 418–425.http://ieeexplore.ieee.org/document/7789549/ Google Scholar
[21]	Tao R, Gavves E, Smeulders A W M. Siamese instance search for tracking[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1420–1429.http://xueshu.baidu.com/s?wd=paperuri%3A%28cd6b46ac6e7a4082d9c7c7732adcc0e1%29&filter=sc_long_sign&tn=SE_xueshusource_2kduw22v&sc_vurl=http%3A%2F%2Fieeexplore.ieee.org%2Fdocument%2F7780527%2F&ie=utf-8&sc_us=7275013320574764237 Google Scholar
[22]	Wang N Y, Li S Y, Gupta A, et al. Transferring rich feature hierarchies for robust visual tracking[J]. arXiv: 1501.04587[cs.CV], 2015. http://www.oalib.com/paper/4069892 Google Scholar
[23]	Zhai M Y, Roshtkhari M J, Mori G. Deep learning of appearance models for online object tracking[J]. arXiv: 1607.02568[cs.CV], 2016. http://www.researchgate.net/publication/305185999_Deep_Learning_of_Appearance_Models_for_Online_Object_Tracking Google Scholar
[24]	王慧燕, 杨宇涛, 张政, 等.深度学习辅助的多行人跟踪算法[J].中国图象图形学报, 2017, 22(3): 349–357. Google Scholar Wang H Y, Yang Y T, Zhang Z, et al. Deep-learning-aided multi-pedestrian tracking algorithm[J]. Journal of Image and Graphics, 2017, 22(3): 349–357. Google Scholar
[25]	王晓冬.视觉角度对游戏可玩性的影响[J].河南科技, 2014(7): 12. Google Scholar
[26]	Horikoshi K, Misawa K, Lang K. 20-fps motion capture of phase-controlled wave-packets for adaptive quantum control[C]//Proceedings of the 15th International Conference on Ultrafast Phenomena XV, 2006: 175–177.http://www.springerlink.com/content/n2217125511444v1/ Google Scholar

Overview

Overview

Overview: Deep learning has achieved good results in image classification, semantic segmentation, target detection and recognition. However, it is still restricted by small sample training sets on object tracking. Object tracking is one of the most important research topics in the field of computer vision, and has a wide range of applications. The challenge of object tracking lies in the complex states such as the target rotation, multi target, blur target, complex background, size change, target occlusion, fast moving and so on. In this paper, based on muti-domain network (MDNet), fast deep learning for aircraft tracking (FDLAT) algorithm is proposed to track aircraft target. This algorithm uses feature-based transfer learning to make up the inferiority of small sample sets, uses specific data sets to update parameters of convolutional layers and fully connected layers, and use it to distinguish aircraft from background. After building the training model, we put the aircraft video sets into the model and tracked the aircraft using regression model and a simple line on-line update, to increase the speed while ensuring the accuracy. This algorithm achieves robust tracking for aircraft in rotation, similar targets, fuzzy targets, complex environment, scale transformation, target occlusion, morphological transformation and other complex states. FDLAT is designed for lifting the speed while guaranteeing the precision of tracking. For the application of aircraft tracking, the FDLAT networks is trained by using 3 convolutional layers (Conv1~Conv3 of VGGNets) to extract the feature of aircraft target. Fc6 is a single layer and Fc4~Fc6 are used for the two classifications of aircraft and background, and the outputs are the probability of aircraft and background. In the process of tracking, the trained networks are used as feed-forward networks, and the candidate box of the maximum score of outputs is regressed to get the target location, while on-line updating is done by a simple linear operation. Our FDLAT algorithm is robust in aircraft target tracking, and basically meets the real-time requirements with high accuracy. This algorithm uses convolutional layers for feature extraction and fully connected layers for classification. Then the outputs of the networks perform a regression and location update operation in the testing process, which has good performance for scaling, occlusion, stealth, interference scene and covers the shortage of MDNet. A speed of 20.36 frames with the overlap reached 0.592 is achieved in the ILSVRC2015 detection sets of aircraft, basically meets the requirement of real-time for aircraft target tracking application.