Chen H Y, Zhao P, Yan H W. Crack detection based on multi-scale Faster RCNN with attention[J]. Opto-Electron Eng, 2021, 48(1): 200112. doi: 10.12086/oee.2021.200112
Citation: Chen H Y, Zhao P, Yan H W. Crack detection based on multi-scale Faster RCNN with attention[J]. Opto-Electron Eng, 2021, 48(1): 200112. doi: 10.12086/oee.2021.200112

Crack detection based on multi-scale Faster RCNN with attention

    Fund Project: National Natural Science Foundation of China (61873315)
More Information
  • The background of the EL image of a photovoltaic cell under electroluminescence (EL) presents complex non-uniform texture features, and there are grain pseudo-defects similar to cracks. At the same time, the cracks appear as multi-scale features with various shapes. The above mentioned difficulties have presented great challenges for the detection task. Therefore, this paper proposes a multi-scale Faster-RCNN model that integrates attention. On the one hand, an improved feature pyramid network is used to obtain multi-scale advanced semantic feature maps to improve the network's feature expression ability of multi-scale crack defects. On the other hand, an improved attention region proposal network A-RPN is adopted to increase the model's attention to crack defects and suppress the characteristics of complex background and grain pseudo-defects. At the same time, in the RPN network training process, a loss function Focal loss is used to reduce the proportion of simple samples in the training process, so that the model pays more attention to the samples that are difficult to distinguish. Experimental results show that this algorithm improves the accuracy of crack defect detection in EL images, reaching nearly 95%.
  • 加载中
  • [1] Anwar S A, Abdullah M Z. Micro-crack detection of multicrystalline solar cells featuring shape analysis and support vector machines[C]//Proceedings of 2012 IEEE International Conference on Control System, Computing and Engineering, 2012: 143-148.

    Google Scholar

    [2] Su B Y, Chen H Y, Zhu Y F, et al. Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor[J]. IEEE Trans Instrum Meas, 2019, 68(12): 4675-4688. doi: 10.1109/TIM.2019.2900961

    CrossRef Google Scholar

    [3] Luo Q W, Sun Y C, Li P C, et al. Generalized completed local binary patterns for time-efficient steel surface defect classification[J]. IEEE Trans Instrum Meas, 2019, 68(3): 667-679. doi: 10.1109/TIM.2018.2852918

    CrossRef Google Scholar

    [4] Tsai D M, Chang C C, Chao S M. Micro-crack inspection in heterogeneously textured solar wafers using anisotropic diffusion[J]. Image Vis Comput, 2010, 28(3): 491-501. doi: 10.1016/j.imavis.2009.08.001

    CrossRef Google Scholar

    [5] Cha Y J, Choi W, Büyüköztürk O. Deep learning‐based crack damage detection using convolutional neural networks[J]. Comput Aided Civ Inf Eng, 2017, 32(5): 361-378. doi: 10.1111/mice.12263

    CrossRef Google Scholar

    [6] Lin H, Li B, Wang X G, et al. Automated defect inspection of LED chip using deep convolutional neural network[J]. J Intell Manuf, 2019, 30(6): 2525-2534. doi: 10.1007/s10845-018-1415-x

    CrossRef Google Scholar

    [7] Duan K W, Bai S, Xie L X, et al. Centernet: keypoint triplets for object detection[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision, 2019: 6568-6577.

    Google Scholar

    [8] Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision, 2017: 2999-3007.

    Google Scholar

    [9] Girshick R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 1440-1448.

    Google Scholar

    [10] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015: 91-99.

    Google Scholar

    [11] Cha Y J, Choi W, Suh G, et al. Autonomous structural visual inspection using region‐based deep learning for detecting multiple damage types[J]. Comput Aided Civ Inf Eng, 2018, 33(9): 731-747. doi: 10.1111/mice.12334

    CrossRef Google Scholar

    [12] 高琳, 陈念年, 范勇. 融合多尺度上下文卷积特征的车辆目标检测[J]. 光电工程, 2019, 46(4): 180331. doi: 10.12086/oee.2019.180331

    CrossRef Google Scholar

    Gao L, Chen N N, Fan Y. Vehicle detection based on fusing multi-scale context convolution features[J]. Opto-Electron Eng, 2019, 46(4): 180331. doi: 10.12086/oee.2019.180331

    CrossRef Google Scholar

    [13] Liu S, Qi L, Qin H F, et al. Path aggregation network for instance segmentation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 8759-8768.

    Google Scholar

    [14] Corbetta M, Shulman G L. Control of goal-directed and stimulus-driven attention in the brain[J]. Nat Rev Neurosci, 2002, 3(3): 201-215. doi: 10.1038/nrn755

    CrossRef Google Scholar

    [15] Frazão M, Silva J A, Lobato K, et al. Electroluminescence of silicon solar cells using a consumer grade digital camera[J]. Measurement, 2017, 99: 7-12. doi: 10.1016/j.measurement.2016.12.017

    CrossRef Google Scholar

    [16] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.

    Google Scholar

    [17] Everingham M, Van Gool L, Williams C K I, et al. The PASCAL visual object classes (VOC) challenge[J]. Int J Comput Vis, 2010, 88(2): 303-338. doi: 10.1007/s11263-009-0275-4

    CrossRef Google Scholar

    [18] Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19.

    Google Scholar

  • Overview: Electroluminescence (EL) images of photovoltaic cells have a non-uniformly textured complex background, and the background contains grain pseudo-defects that are highly similar to the crack structure. At the same time, the cracks are characterized by various sizes and shapes. Existing target detection algorithms based on convolutional neural networks cannot adapt to the above problems. From the perspective of suppressing interference from complex background and improving the adaptability of the model to multi-scale crack defect detection, this paper proposes a multi-scale Faster RCNN model that integrates attention. In photovoltaic cell EL images, the scale of the cracks varies greatly, including a large number of small target cracks. In order to improve the network's ability to express multi-scale crack defects, a path aggregation feature pyramid network (PA-FPN) is proposed. Based on the combination of the residual network ResNet50 and the feature pyramid network FPN, PA-FPN adds a bottom-up path to fuse features. PA-FPN effectively retains shallow feature information, which improves the model's adaptability to multi-scale cracks in EL images and especially the detection results of small-scale cracks. In order to improve the model's attention to crack defects and suppress the characteristics of complex background and grain pseudo-defects, this paper proposes a regional recommendation network A-RPN that incorporates convolutional block attention module (CBAM). CBAM is composed of a channel attention module and a spatial attention module. In this paper, it is experimentally verified that the detection result of the RPN network fused with CBAM is better than that of using an attention modules alone. K-means clustering is used to cluster the crack sizes in the data set to guide the RPN to set the anchor box closer to the actual crack size, which improves the speed and accuracy of the target box regression in the defect detection process. In addition, in the RPN network training process, the loss function Focal loss is used to replace the original cross-entropy loss function, so as to reduce the proportion of simple samples in the training process and make the model pay more attention to the samples that are difficult to distinguish. The entire network can achieve end-to-end training. In order to verify the effectiveness of the improved algorithm, the performance of the original Faster RCNN model, RetinaNet, and CenterNet on multi-scale crack detection of EL images is compared. Through training and testing of 1024 pixels×1024 pixels of photovoltaic cell EL images, experimental results show that the improved Faster RCNN is better than the above mentioned target detection algorithms in accuracy, and has good robustness to the strip-shaped multi-scale cracks, which can be adapted to the EL image with changing complex background.

  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(8)

Tables(4)

Article Metrics

Article views(6935) PDF downloads(2873) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint