基于全卷积神经网络与条件随机场的车道识别方法

叶子豪, 孙锐, 王慧慧. 基于全卷积神经网络与条件随机场的车道识别方法[J]. 光电工程, 2019, 46(2): 180274. doi: 10.12086/oee.2019.180274
引用本文: 叶子豪, 孙锐, 王慧慧. 基于全卷积神经网络与条件随机场的车道识别方法[J]. 光电工程, 2019, 46(2): 180274. doi: 10.12086/oee.2019.180274
Ye Zihao, Sun Rui, Wang Huihui. Lane recognition method based on fully convolution neural network and conditional random fields[J]. Opto-Electronic Engineering, 2019, 46(2): 180274. doi: 10.12086/oee.2019.180274
Citation: Ye Zihao, Sun Rui, Wang Huihui. Lane recognition method based on fully convolution neural network and conditional random fields[J]. Opto-Electronic Engineering, 2019, 46(2): 180274. doi: 10.12086/oee.2019.180274

基于全卷积神经网络与条件随机场的车道识别方法

  • 基金项目:
    国家自然科学基金项目(61471154);安徽省科技攻关科技项目(170d0802181)
详细信息
    作者简介:
    通讯作者: 孙锐(1976-),男,博士,教授,主要从事计算机视觉与机器学习的研究。E-mail:sunrui@hfut.edu.cn
  • 中图分类号: TP301.6

Lane recognition method based on fully convolution neural network and conditional random fields

  • Fund Project: Supported by National Natural Science Foundation of China (61471154) and Anhui Science and Technology Research Project (170d0802181)
More Information
  • 本文针对传统车道识别方法在复杂路面中自适应能力差的特点,基于图像分割技术提出了一种基于全卷积神经网络与条件随机场的车道识别方法。该方法通过大量数据的训练,使神经网络模型可以识别出车道,并且再通过条件随机场使得分割出来的车道覆盖面积及车道边缘的处理更加完善。同时,本文为了解决高速公路中对检测实时性的高要求,设计了一个全卷积神经网络,该网络结构简单,只有13万个参数,并且做出如下三点改进:采用BN算法提高网络的泛化能力及收敛速度;采用了LeakyReLU激活函数取代了一般使用的relu或者sigmoid激活函数,并且采用Nadam作为网络的优化器使得该网络具有更好的鲁棒性;采用条件随机场作为后端处理解决车道边缘处分割不足并且加大了车道覆盖面积。最后本文为了解决城市道路检测中道路环境复杂的问题,利用FCN-16s网络模型加条件随机场的后端处理实现了复杂城市道路的识别。实验证明,在面对高速公路的高速及车道简单环境下,本文设计的网络模型更具有实时性且足够胜任车道的识别。在面对城市道路的复杂环境下,FCN-16s模型加条件随机场更能精确地识别出车道,并在KITTI道路检测基准上取得不错的结果。

  • Overview: In recent years, with the rapid development of smart cars, autonomous driving has attracted great attention from industry and academia. Lane recognition is a fundamental work for achieving automatic driving. Specifically, accurate lane recognition not only allows the vehicle to travel on the correct road but also alerts the control system with other information such as lane markings, pedestrians and other anomalous events. The traditional lane detection method generally adopts the steps of preprocessing, edge detection, Hough transform, lane matching, lane segmentation, etc. These methods interact with each other, and it is difficult to achieve global optimization and real-time optimization simultaneously. Furthermore, the lane change adaptation is obviously insufficient. Deep neural network is a powerful visual analysis tool. Compared with the traditional shallow computing structure, its main advantage lies in its self-learning ability according to input data and its end-to-end unified structure. Aiming at the poor adaptability of traditional lane recognition method in complex pavement, this paper proposes a lane recognition method based on full convolutional neural network and conditional random field, according to image segmentation technology. The method can make the neural network model identify the lanes by training a large amount of data, and then make the segmentation of the lanes' coverage and the lane edges more perfect through the conditional random field. At the same time, in order to solve the high requirement of real-time detection in expressway, a fully convolution neural network is designed in this paper. The network structure is simple with only 130000 parameters and three improvements are made as follows: BN algorithm is used to improve network generalization Ability and convergence rate; LeakyReLU activation function is used to replace the commonly used relu or sigmoid activation function, and using Nadam as the network optimizer makes the network have better robustness. Conditional random field is used as the back-end processing solution insufficient lane segmentation and further to increase lane coverage. Finally, in order to solve the problem of complex road environment in urban road testing, this paper uses the back-end processing of FCN-16s network model and conditional random field to realize the recognition of complex urban roads. Experiments show that the network model designed in this paper is more real-time and sufficient for lane identification in the face of high-speed expressways and simple lanes. In the complex environment of urban road, FCN-16s model plus conditional random field can identify lane more accurately and get good result on KITTI road test benchmarks.

  • 加载中
  • 图 1  全卷积神经网络原理图

    Figure 1.  Schematic diagram of the full convolutional neural network

    图 2  VGG16模型图

    Figure 2.  VGG16 model diagram

    图 3  车道识别流程

    Figure 3.  Lane recognition process

    图 4  全卷积神经网络结构

    Figure 4.  Full convolutional neural network structure

    图 5  直线公路对比。(a) FCN;(b) FCN+CRF;(c)真实场景

    Figure 5.  Straight road comparison. (a) FCN; (b) FCN+CRF; (c) Real scene

    图 6  损坏公路对比。(a) FCN;(b) FCN+CRF;(c)真实场景

    Figure 6.  Corrupted road comparison. (a) FCN; (B) FCN+CRF; (c) Real scene

    图 7  弯道公路对比。(a) FCN;(b) FCN+CRF;(c)真实场景

    Figure 7.  Curve road comparison. (a) FCN; (B) FCN+CRF; (c) Real scene

    图 8  车道线数据集下的车道线标注

    Figure 8.  Lane line labeling under the lane line data set

    图 9  UU_ROAD_000020检测对比。(a) FCN-LC; (b) ANN;(c) BM;(d) FCN+CRF

    Figure 9.  UU_ROAD_000020 detection comparison. (a) FCN-LC; (b) ANN; (c) BM; (d) FCN+CRF

    图 10  UU_ROAD_000082检测对比。(a) FCN-LC; (b) ANN;(c) BM;(d) FCN+CRF

    Figure 10.  Detection comparison of 10 UU_ROAD_000082. (a) FCN-LC; (b) ANN; (c) BM; (d) FCN+CRF

    表 1  车道线检测方法的性能FP指标对比

    Table 1.  Comparison of performance of FP indicators among lane line detection methods

    Baseline ReNet[20] DenseCRF[21]
    Normal 83.1 83.3 81.3
    Crowded 61.0 60.5 58.8
    Night 56.9 56.3 54.2
    No line 34.0 34.5 31.9
    Shadow 54.7 55.0 56.3
    Arrow 74.0 74.1 71.2
    Dazzle light 49.9 48.2 46.2
    Curve 61.0 59.9 57.8
    Crossroad 2060 2296 2253
    Total 63.2 62.9 61.0
    下载: 导出CSV

    表 2  KITTI数据集信息

    Table 2.  KITTI data set information

    场景分类 训练集 测试集
    UU 98 100
    UM 95 96
    UMM 96 94
    城市道路总数 289 290
    下载: 导出CSV

    表 3  UU测试集下FCN-16s与FCN+CRF算法对比

    Table 3.  Comparison between FCN-16s and FCN+CRF algorithms under UU test set

    算法模型 MaxF/% PRE/% REC/%
    FCN-16s[9] 82.03 80.64 83.41
    FCN+CRF 89.87 88.93 90.11
    下载: 导出CSV

    表 4  UU测试集下的算法对比

    Table 4.  Comparison among algorithms under the UU test set

    算法模型 MaxF/% AP/% PRE/% REC/% FPR/% FNR/% Runtime/s
    FCN-LC[7] 86.27 75.37 86.65 85.89 4.31 14.11 0.03
    ANN[6] 54.07 36.61 39.28 86.69 43.67 13.31 3
    BM[9] 78.43 62.46 70.87 87.80 11.76 12.20 2
    DDN[10] 91.76 86.84 93.06 90.50 2.20 9.50 2
    FCN+CRF 89.87 82.96 88.93 90.11 4.22 9.04 0.08
    下载: 导出CSV
  • [1]

    李明, 黄华, 夏建刚.基于Hough变换的车道检测改进算法研究[J].计算机工程与设计, 2012, 33(4):1638-1642. doi: 10.3969/j.issn.1000-7024.2012.04.075

    Li M, Huang H, Xia J G. Research of robust lane detection algorithm based on Hough transform[J]. Computer Engineering And Design, 2012, 33(4):1638-1642. doi: 10.3969/j.issn.1000-7024.2012.04.075

    [2]

    Sharifi M, Fathy M, Mahmoudi M T. A classified and comparative study of edge detection algorithms[C]//Proceedings of International Conference on Information Technology: Coding and Computing, Las Vegas, NV, USA, 2002: 117.

    [3]

    Jung H G, Lee Y H, Kang H J, et al. Sensor fusion-based lane detection for LKS+ACC system[J]. International Journal of Automotive Technology, 2009, 10(2): 219-228. doi: 10.1007/s12239-009-0026-0

    [4]

    McCall J C, Trivedi M M. An integrated, robust approach to lane marking detection and lane tracking[C]//IEEE Intelligent Vehicles Symposium, Parma, Italy, 2004: 533-537.

    [5]

    郭磊, 李克强, 王建强, 等.应用方向可调滤波器的车道线识别方法[J].机械工程学报, 2008, 44(8): 214-218, 226. doi: 10.3321/j.issn:0577-6686.2008.08.038

    [6]

    Vitor G B, Lima D A, Victorino A C, et al. A 2D/3D vision based approach applied to road detection in urban environments[C]//2013 IEEE Intelligent Vehicles Symposium, Gold Coast, QLD, Australia, 2013: 952-957.

    [7]

    Mendes C C T, Frémont V, Wolf D F. Exploiting fully convolutional neural networks for fast road detection[C]//2016 IEEE International Conference on Robotics and Automation, Stockholm, Sweden, 2016.

    [8]

    Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683

    [9]

    Wang B, Frémont V, Rodríguez S A. Color-based road detection and its evaluation on the KITTI road benchmark[C]//2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, 2014: 31-36.

    [10]

    Mohan R. Deep deconvolutional networks for scene parsing[EB/OL]. https://available:arxiv.org/abs/1411.4101.

    [11]

    Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. https: //available: arxiv.org/abs/1409.1556.

    [12]

    Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, 2012: 1097-1105.

    [13]

    Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 1-9.

    [14]

    Brust C A, Sickert S, Simon M, et al. Convolutional patch networks with spatial prior for road detection and urban scene understanding[C]//VISAPP 2015-10th International Conference on Computer Vision Theory and Applicayions, Berlin, Germany, 2015: 510-517.

    [15]

    Mendes C C T, Frémont V, Wolf D F. Exploiting fully convolutional neural networks for fast road detection[C]//2016 IEEE International Conference on Robotics and Automation, Stockholm, Sweden, 2016.

    [16]

    Dinesh Reddy N, Prateek Singhal, Madhava Krishna K. Semantic Motion Segmentation Using Dense CRF Formulation[C]//Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP, Bangalore, India.2014: a56.

    [17]

    Fritsch J, Kühnl T, Geiger A. A new performance measure and evaluation benchmark for road detection algorithms[C]//16th International IEEE Conference on Intelligent Transportation Systems, The Hague, Netherlands, 2013: 1693-1700.

    [18]

    Dan C C, Giusti A, Gambardella L M, et al. Deep neural networks segment neuronal membranes in electron microscopy images[J]. Advances in Neural Information Processing Systems, 2012, 25: 2852-2860. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CC0213884171

    [19]

    Dai J F, He K M, Sun J. Convolutional feature masking for joint object and stuff segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2014: 3992-4000.

    [20]

    Huval B, Wang T, Tandon S, et al. An empirical evaluation of deep learning on highway driving[EB/OL].https://arxiv.org/pdf/1504.01716.pdf.

    [21]

    Krähenbühl P, Koltun V. Efficient inference in fully connected CRFs with Gaussian edge potentials[C]//Proceedings of the 24th Neural Information Processing Systems, Granada, Spain, 2011: 109-117.

    [22]

    孙锐, 方蔚, 高隽.暗通道和测度学习的雾天行人再识别[J].光电工程, 2016, 43(12): 142-146. doi: 10.3969/j.issn.1003-501X.2016.12.022

    Sun R, Fang W, Gao J. Person re-identification in foggy weather based on dark channel prior and metric learning[J]. Opto-Electronic Engineering, 2016, 43(12): 142-146. doi: 10.3969/j.issn.1003-501X.2016.12.022

  • 加载中

(10)

(4)

计量
  • 文章访问数:  7096
  • PDF下载数:  3242
  • 施引文献:  0
出版历程
收稿日期:  2018-05-23
修回日期:  2018-07-12
刊出日期:  2019-02-18

目录

/

返回文章
返回