Improvement of GBS-YOLOv7t for steel surface defect detection

Liang Liming; Long Pengwei; Lu Baohe; Li Renjie

doi:10.12086/oee.2024.240044

Article navigation > Opto-Electronic Engineering > 2024 Vol. 51 > No. 5 > 240044

Next Article Previous Article

Liang L M, Long P W, Lu B H, et al. Improvement of GBS-YOLOv7t for steel surface defect detection[J]. Opto-Electron Eng, 2024, 51(5): 240044. doi: 10.12086/oee.2024.240044

Citation:

Liang L M, Long P W, Lu B H, et al. Improvement of GBS-YOLOv7t for steel surface defect detection[J]. Opto-Electron Eng, 2024, 51(5): 240044. doi: 10.12086/oee.2024.240044

Improvement of GBS-YOLOv7t for steel surface defect detection

School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou, Jiangxi 341000, China

Fund Project: Project supported by National Natural Science Foundation of China (51365017, 61463018), Jiangxi Provincial Natural Science Foundation (20192BAB205084), Jiangxi Provincial Department of Education Scientific and Technological Research Key Project (GJJ170491), and Jiangxi Provincial Postgraduate Innovation Special Funds Project (YC2022-S676)

More Information

^*Corresponding author: 2637018663@qq.com

Received Date 29 February 2024

Revised Date 20 March 2024

Accepted Date 22 March 2024

Published Date 25 May 2024

Abstract

Abstract

Given that small targets are predominant in the steel surface defect areas, most existing methods cannot balance the trade-off between detection accuracy and speed. In this paper, we propose a steel surface defect detection algorithm based on YOLOv7-tiny (GBS-YOLOv7t). Firstly, we design the GAC-FPN network to fully integrate the target semantic information progressively and across layers, aiming to address the limited information flow issue in traditional feature pyramids. Secondly, we embed a bi-level routing attention (BRA) module to endow the model with dynamic query and sparse perception capabilities, thus enhancing the detection accuracy of small targets. Thirdly, we introduce the SIoU loss function to improve the training and inference capabilities of the model, and to enhance the network robustness. Experimental validation on the public dataset NEU-DET demonstrates an mAP of 72.9% and a precision of 69.9% for GBS-YOLOv7t, achieving improvements of 4.2% and 8.5%, respectively, over the original YOLOv7-tiny model. The FPS reaches 104.1 frames, indicating strong real-time performance. Compared to other detection algorithms, GBS-YOLOv7t is more effective in detecting small targets in steel surface areas, with experimental results showing that the improved algorithm better balances the detection accuracy and speed.
- defect detection /
- YOLOv7-tiny /
- GAC-FPN network /
- bi-level routing attention /
- SIoU

FullText(HTML)

References

[1]	陈海永, 赵鹏, 闫皓炜. 融合注意力的多尺度Faster RCNN的裂纹检测[J]. 光电工程, 2021, 48(1): 200112. doi: 10.12086/oee.2021.200112 CrossRef Google Scholar Chen H Y, Zhao P, Yan H W. Crack detection based on multi-scale Faster RCNN with attention[J]. Opto-Electron Eng, 2021, 48(1): 200112. doi: 10.12086/oee.2021.200112 CrossRef Google Scholar
[2]	梁礼明, 卢宝贺, 龙鹏威, 等. 自适应特征融合级联Transformer视网膜血管分割算法[J]. 光电工程, 2023, 50(10): 230161. doi: 10.12086/oee.2023.230161 CrossRef Google Scholar Liang L M, Lu B H, Long P W, et al. Adaptive feature fusion cascade transformer retinal vessel segmentation algorithm[J]. Opto-Electron Eng, 2023, 50(10): 230161. doi: 10.12086/oee.2023.230161 CrossRef Google Scholar
[3]	赵朗月, 吴一全. 基于机器视觉的表面缺陷检测方法研究进展[J]. 仪器仪表学报, 2023, 43(1): 198−219. doi: 10.19650/j.cnki.cjsi.J2108805 CrossRef Google Scholar Zhao L Y, Wu Y Q. Research progress of surface defect detection methods based on machine vision[J]. Chin J Sci Instrum, 2023, 43(1): 198−219. doi: 10.19650/j.cnki.cjsi.J2108805 CrossRef Google Scholar
[4]	Liu Z, Yeoh J K W, Gu X Y, et al. Automatic pixel-level detection of vertical cracks in asphalt pavement based on GPR investigation and improved mask R-CNN[J]. Autom Constr, 2023, 146: 104689. doi: 10.1016/j.autcon.2022.104689 CrossRef Google Scholar
[5]	Zhang W M, Zhu Q K, Li Y Q, et al. MAM Faster R-CNN: improved Faster R-CNN based on malformed attention module for object detection on X-ray security inspection[J]. Digital Signal Process, 2023, 139: 104072. doi: 10.1016/j.dsp.2023.104072 CrossRef Google Scholar
[6]	Hu B, Wang J H. Detection of PCB surface defects with improved faster-RCNN and feature pyramid network[J]. IEEE Access, 2020, 8: 108335−108345. doi: 10.1109/ACCESS.2020.3001349 CrossRef Google Scholar
[7]	Qian H M, Wang H L, Feng S, et al. FESSD: SSD target detection based on feature fusion and feature enhancement[J]. J Real-Time Image Process, 2023, 20(1): 2. doi: 10.1007/s11554-023-01258-y CrossRef Google Scholar
[8]	Hussain M. YOLO-v1 to YOLO-v8, the rise of YOLO and its complementary nature toward digital manufacturing and industrial defect detection[J]. Machines, 2023, 11(7): 677. doi: 10.3390/machines11070677 CrossRef Google Scholar
[9]	Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 936–944. https://doi.org/10.1109/CVPR.2017.106. Google Scholar
[10]	Liu S, Qi L, Qin H F, et al. Path aggregation network for instance segmentation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018: 8759–8768. https://doi.org/10.1109/CVPR.2018.00913. Google Scholar
[11]	方钧婷, 谭晓阳. 注意力级联网络的金属表面缺陷检测算法[J]. 计算机科学与探索, 2021, 15(7): 1245−1254. doi: 10.3778/j.issn.1673-9418.2007005 CrossRef Google Scholar Fang J T, Tan X Y. Defect detection of metal surface based on attention cascade R-CNN[J]. J Front Comput Sci Technol, 2021, 15(7): 1245−1254. doi: 10.3778/j.issn.1673-9418.2007005 CrossRef Google Scholar
[12]	熊聪, 于安宁, 高兴华, 等. 基于改进YOLOX的钢材表面缺陷检测算法[J]. 电子测量技术, 2023, 46(9): 151−157. doi: 10.19651/j.cnki.emt.2211012 CrossRef Google Scholar Xiong C, Yu A N, Gao X H, et al. Steel surface defect detection algorithm based on improved YOLOX[J]. Electron Meas Technol, 2023, 46(9): 151−157. doi: 10.19651/j.cnki.emt.2211012 CrossRef Google Scholar
[13]	赵春华, 罗顺, 谭金铃, 等. 基于PC-YOLOv7算法钢材表面缺陷检测[J]. 国外电子测量技术, 2023, 42(9): 137−145. doi: 10.19652/j.cnki.femt.2304998 CrossRef Google Scholar Zhao C H, Luo S, Tan J L, et al. Detection of steel surface defects based on PC-YOLOv7 algorithm[J]. Foreign Electron Meas Technol, 2023, 42(9): 137−145. doi: 10.19652/j.cnki.femt.2304998 CrossRef Google Scholar
[14]	Lian J W, He J H, Niu Y, et al. Fast and accurate detection of surface defect based on improved YOLOv4[J]. Assem Autom, 2022, 42(1): 134−146. doi: 10.1108/AA-04-2021-0044 CrossRef Google Scholar
[15]	马燕婷, 赵红东, 阎超, 等. 改进YOLOv5网络的带钢表面缺陷检测方法[J]. 电子测量与仪器学报, 2022, 36(8): 150−157. doi: 10.13382/j.jemi.B2205354 CrossRef Google Scholar Ma Y T, Zhao H D, Yan C, et al. Strip steel surface defect detection method by improved YOLOv5 network[J]. J Electron Meas Instrum, 2022, 36(8): 150−157. doi: 10.13382/j.jemi.B2205354 CrossRef Google Scholar
[16]	Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 2023: 7464–7475. https://doi.org/10.1109/CVPR52729.2023.00721. Google Scholar
[17]	Yang G Y, Lei J, Zhu Z K, et al. AFPN: Asymptotic feature pyramid network for object detection[C]//2023 IEEE International Conference on Systems, Man, and Cybernetics, Honolulu, 2023: 2184–2189. https://doi.org/10.1109/SMC53992.2023.10394415. Google Scholar
[18]	Li H L, Li J, Wei H B, et al. Slim-neck by GSConv: A better design paradigm of detector architectures for autonomous vehicles[Z]. arXiv: 2206.02424, 2022. https://doi.org/10.48550/arXiv.2206.02424. Google Scholar
[19]	Liu S T, Huang D, Wang Y H. Learning spatial fusion for single-shot object detection[Z]. arXiv: 1911.09516, 2019. https://doi.org/10.48550/arXiv.1911.09516. Google Scholar
[20]	Zhu L, Wang X J, Ke Z H, et al. BiFormer: Vision transformer with bi-level routing attention[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 2023: 10323–10333. https://doi.org/10.1109/CVPR52729.2023.00995. Google Scholar
[21]	Gevorgyan Z. SIoU loss: More powerful learning for bounding box regression[Z]. arXiv: 2205.12740, 2022. https://doi.org/10.48550/arXiv.2205.12740. Google Scholar
[22]	Li Z G, Wei X M, Hassaballah M, et al. A deep learning model for steel surface defect detection[J]. Complex Intell Syst, 2024, 10(1): 885−897. doi: 10.1007/s40747-023-01180-7 CrossRef Google Scholar
[23]	曹义亲, 伍铭林, 徐露. 基于改进YOLOv5算法的钢材表面缺陷检测[J]. 图学学报, 2023, 44(2): 335−345. doi: 10.11996/JG.j.2095-302X.2023020335 CrossRef Google Scholar Cao Y Q, Wu M L, Xu L. Steel surface defect detection based on improved YOLOv5 algorithm[J]. J Graphics, 2023, 44(2): 335−345. doi: 10.11996/JG.j.2095-302X.2023020335 CrossRef Google Scholar

Overview

Overview

Aiming at the problem that most of the existing methods are unable to equalize the detection accuracy and the speed because of the predominance of small targets in the defective region of the steel surface, this paper proposes a steel surface defect detection algorithm based on YOLOv7-tiny (GBS-YOLOv7t). The method, firstly, takes into account that the feature fusion network of the original YOLOv7-tiny model adopts the traditional path aggregation network (PANet), which is designed with a bottom-up structure, but the bottom-up structure will have the problem of limiting the information flow. To address this problem, this paper compresses the model complexity and further preserves the semantic information of small targets by introducing the asymptotic feature pyramid (AFPN), and on its basis, by introducing the ghost shuffle mixing convolution (GSConv) and cross-layer connectivity. Based on the above improvements, the Ghost Asymptotic Cross-layer Fusion Network (GAC-FPN) is designed and replaces the original YOLOv7-tiny path aggregation network. The GAC-FPN network adopts an asymptotic and cross-layer approach to fully fuse the semantic information of the target features, which effectively improves the problem of restricting the flow of information in the top-down structure in the traditional feature pyramid. Secondly, to increase the model's accuracy in detecting the small targets. To improve the detection accuracy of the model for small targets, a Bi-Level Routing Attention module is embedded in the backbone network, and the optimal location of the module in the backbone network is verified through experiments, and the results show that the module makes the model possess the ability of dynamic querying and sparsity perception while taking into account the number of network parameters and the computational complexity, which effectively improves the detection accuracy of the model for small targets; thirdly, a SIoU loss function is introduced to replace the CIoU loss function of the original network, effectively improving the model training and reasoning ability, which improves the model training and inference ability, and enhances the network robustness. Finally, experimental validation is carried out on the publicly available Northeastern University Steel Surface Defect Dataset (NEU-DET), and the experimental results show that the mAP and accuracy of the GBS-YOLOv7t algorithm reach 72.9% and 69.9%, respectively, which are improved by 4.2% and 8.5%, respectively, compared with the original model of YOLOv7-tiny; the FPS reaches 104.1 frames, which is strong real-time performance. Compared with other classical detection models and current mainstream algorithms, the GBS-YOLOv7t algorithm has better performance and is more effective in detecting small targets on the surface area of steel, and the experiments show that the improved algorithm better balances lightweight, detection accuracy and speed.