E-mail Alert
RSS
| Citation: |
|
Deblurring algorithm based on pedestrian abnormal behavior generation countermeasure network
-
Abstract
To solve the problem of motion blur in abnormal behavior detection, a fast motion blur removal algorithm, based on DeblurGAN, is proposed. Three 3×3 convolutions are used to replace the 7×7 convolution in the original generator. The transposed convolution is discarded. Firstly, bilinear interpolation is used to expand the size of the feature map which needs upsampling. The residual unit is replaced by a residual density block (RRDB) in the original algorithm. The RRDB is then scaled to 0~1 to avoid unstable training. The L1 loss of gradient images is added to the loss function of the original generator. As the DeblurGAN reconstructed image edge is often not clear enough, the edge information of the image is added to make the reconstructed image edge more obvious. The effectiveness of this method is verified by experiments and is compared with other similar algorithms like DeblurGAN. The PSNR of the optimized model is improved by 0.94. The structure similarity and speed are equivalent. The chessboard lattice problem in the reconstructed image is solved. The edge of detail is more prominent. The performance of the proposed model is better than that of other related algorithms. -
-
References
[1] 鄢化彪, 黄绿娥, 王鹏. 高速运动的轨道图像去模糊[J]. 铁道学报, 2019, 41(12): 75-80. doi: 10.3969/j.issn.1001-8360.2019.12.010 Yan H B, Huang L E, Wang P. Motion deblurring from fast railway images[J]. J China Railway Soc, 2019, 41(12): 75-80. doi: 10.3969/j.issn.1001-8360.2019.12.010 [2] Sun J, Cao W F, Xu Z B, et al. Learning a convolutional neural network for non-uniform motion blur removal[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, 2015: 769-777. [3] Nimisha T M, Singh A K, Rajagopalan A N. Blur-invariant deep learning for blind-deblurring[C]//Proceedings of the IEEE International Conference on Computer Vision, Venice, 2017: 4762-4770. [4] Ramakrishnan S, Pachori S, Gangopadhyay A, et al. Deep generative filter for motion deblurring[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, 2017: 2993-3000. [5] Shao W Z, Liu Y Y, Ye L Y, et al. DeblurGAN+: Revisiting blind motion deblurring using conditional adversarial networks[J]. Signal Processing, 2020, 168: 107338. doi: 10.1016/j.sigpro.2019.107338 [6] Wang X T, Ke Y, Wu S X, et al. ESRGAN: enhanced super-resolution generative adversarial networks[C]//European Conference on Computer Vision, Munich, 2018: 6379. [7] 孙季丰, 朱雅婷, 王恺. 基于DeblurGAN和低秩分解的去运动模糊[J]. 华南理工大学学报(自然科学版), 2020, 48(1): 32-41, 50. Sun J F, Zhu Y T, W K. Motion deblurring based on DeblurGAN and low rank decomposition[J]. J South China Univ Technol (Nat Sci), 2020, 48(1): 32-41, 50. [8] Yu J, Chang Z C, Xiao C B, et al. Blind image deblurring based on sparse representation and structural self-similarity[C]//2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, 2017: 1328-1332. [9] 唐述, 龚卫国, 仲建华. 稀疏平滑特性的多正则化约束图像盲复原方法[J]. 软件学报, 2013, 24(5): 1143-1154. Tang S, Gong W G, Zhong J H. Sparsity and smoothing multi-regularization constraints for blind image deblurring[J]. J Softw, 2013, 24(5): 1143-1154. [10] 赵树阳, 李建武. 基于生成对抗网络的低秩图像生成方法[J]. 自动化学报, 2018, 44(5): 829-839. Zhao S Y, Li J W. Generative adversarial network for generating low-rank images[J]. Acta Automat Sin, 2018, 44(5): 829-839. [11] Li C, Wand M. Precomputed real-time texture synthesis with markovian generative adversarial networks[C]//Proceedings of European Conference on Computer Vision, Amsterdam, 2016: 702-716. [12] Gulrajani I, Ahmend F, Arjovsky M, et al. Improved training of Wassertstein GANs[EB/OL]. (2017-12-25)[2018-08-15]. https://arxiv.org/pdf/1704.00028.pdf. [13] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10) [2018-08-17]. https://arxiv.org/pdf/1409.1556.pdf. [14] Nah S, Kim T H, Lee K M. Deep multiscale convolutional neural network for dynamic scene deblurring[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 257-265. [15] Redmon J, Farhadi A. YOLOv3: An Incremental Improvement[Z]. arXiv: 1804.02767, 2018. [16] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. J Mach Learn Res, 2014, 15(1): 1929-1958. [17] 赵增顺, 高寒旭, 孙骞. 生成对抗网络理论框架、衍生模型与应用最新进展[J]. 小型微型计算机系统, 2018, 39(12): 2602-2606. doi: 10.3969/j.issn.1000-1220.2018.12.008 Zhao Z S, Gao H X, Sun Q. Latest development of the theory framework, derivative model and application of genera-tive adversarial nets[J]. J Chin Comput Syst, 2018, 39(12): 2602-2606. doi: 10.3969/j.issn.1000-1220.2018.12.008 [18] 吴迪, 赵洪田, 郑世宝. 密集连接卷积网络图像去模糊[J]. 中国图象图形学报, 2020, 25(5): 890-899. Wu D, Zhao H T, Zheng S B. Motion deblurring method based on DenseNets[J]. J Image Graph, 2020, 25(5): 890-899. [19] Zhang Y L, Tian Y P, Kong Y, et al. Residual dense network for image super-resolution[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 2472-2481. [20] 秦绪佳, 柯玲玲, 范颖琳, 等. 改进的超拉普拉斯约束的单幅图像去模糊算法[J]. 小型微型计算机系统, 2018, 39(5): 1097-1102. doi: 10.3969/j.issn.1000-1220.2018.05.041 Qin X J, Ke L L, Fan Y L, et al. Improved hyper-laplacian constraint single image deblurring algorithm[J]. J Chin Comput Syst, 2018, 39(5): 1097-1102. doi: 10.3969/j.issn.1000-1220.2018.05.041 [21] Zhou L H, Min W D, Lin D Y, et al. Detecting motion blurred vehicle logo in IoV Using Filter-DeblurGAN and VL-YOLO[J]. IEEE Trans Veh Technol, 2020, 69(4): 3604-3614. doi: 10.1109/TVT.2020.2969427 [22] Truong N Q, Lee Y W, Owais M, et al. SlimDeblurGAN-based motion deblurring and marker detection for autonomous drone landing[J]. Sensors, 2020, 20(14): 3918. doi: 10.3390/s20143918 [23] Kupyn O, Martyniuk T, Wu J R, et al. DeblurGAN-v2: deblurring (Orders-of-Magnitude) faster and better[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 2019: 8877-8886. -
Overview
Overview: In a complex environment, such as motion, defocusing and other noise, the image quality will become worse, and it will increase the difficulty of detection and recognition. Traditional deblurring methods usually estimate the motion blur kernel according to the prior assumption of clear images, and then convolute with the blurred image and remove the blur with additive noise. However, the estimation of motion blur kernel is complex, and the prior assumption of a clear image is not generalized. As a result, it is difficult to achieve the goal. Deep-learning based motion blur removal methods are constantly proposed. In order to solve the problem of motion blur in abnormal behavior detection, a fast motion blur removal algorithm, based on DeblurGAN, is proposed. In order to further improve the detail clarity of deblurring and solve the chessboard effect of the original algorithm (inserting pixel 0 into each row and column of the feature map pixel which causes the reconstructed image pixels unevenly distributed). The transposed convolution is discarded. Firstly, bilinear interpolation is used to expand the size of the feature map which needs upsampling. On the premise of the same receptive field, three 3×3 convolutions are used to replace the 7×7 convolution in the original generator. As a result, the parameter is reduced and the nonlinearity of the network is increased. In order to solve the layered feature of the original algorithm due to the loss of residual cells, the residual unit is replaced by a residual density block (RRDB) in the original algorithm. The RRDB is then scaled to 0~1 to avoid unstable training. As a result, the details of the restored image are enriched. In addition, in order to solve the problem that the edge of the reconstructed image is not clear, the L1 loss of gradient images is added to the loss function of the original generator. The edge information of the image is added to make the reconstructed image edge more obvious. The effectiveness of this method is verified by experiments and is compared with other similar algorithms, such as DeblurGAN. The PSNR of the optimized model is improved by 0.94. The structure similarity and speed are equivalent. The chessboard lattice problem in the reconstructed image is solved. The edge information is more prominent. The performance of the model is better than that other related algorithms. The improved algorithm of Tiny YOLOv3 is used to verify the abnormal behavior of the escalator after deblurring. It is found that the detection of accuracy and recall rate are improved by 8% and 9% respectively after deblurring, which is helpful to improve the detection accuracy of abnormal behavior of pedestrians on the escalator in the real scene.
-
Access History
Figures(10)
Tables(2)
Article Metrics
Export File
Citation
Format
Content
DownLoad:
-
Figure 1.
Algorithmic framework based on cGAN
-
Figure 2.
Original generator structure
-
Figure 3.
Generator structure
-
Figure 4.
Residual structures of two methods. (a) Residual unit; (b) Residual-in-residual dense block
-
Figure 5.
Two upsampling methods. (a) Transpose convolution; (b) Bilinear interpolation
-
Figure 6.
Dense block
-
Figure 7.
Discriminator structure
-
Figure 8.
Comparison of results. (a) Blurred image; (b) DeblurGAN method; (c) Method in the Ref.[14]; (d) Method in the Ref.[18]; (e) Our algorithm
-
Figure 9.
Comparison of results. (a) Blurred image; (b) DeblurGAN method; (c) Method in the Ref.[14]; (d) Method in the Ref.[18]; (e) Our algorithm
-
Figure 10.
Detection results. (a) Blurred image; (b) DeblurGAN method; (c) Method in the Ref.[14]; (d) Method in the Ref.[18]; (e) Our algorithm
