Chen M H, Wang T, Yuan Y, et al. Study on retinal OCT segmentation with dual-encoder[J]. Opto-Electron Eng, 2023, 50(10): 230146. doi: 10.12086/oee.2023.230146
Citation: Chen M H, Wang T, Yuan Y, et al. Study on retinal OCT segmentation with dual-encoder[J]. Opto-Electron Eng, 2023, 50(10): 230146. doi: 10.12086/oee.2023.230146

Study on retinal OCT segmentation with dual-encoder

    Fund Project: Project supported by Shanghai Science and Technology Commission Industry-university-research Medical Project (15DZ1940400)
More Information
  • There are noises and speckles in OCT retinal images, and a single extraction of spatial features is often easy to miss some important information. Therefore, the target region cannot be accurately segmented. OCT images themselves have spectral frequency domain characteristics. Aiming at the frequency domain characteristics of OCT images, this paper proposes a new dual encoder model based on U-Net and fast Fourier convolution to improve the segmentation performance of the retinal layer and liquid in OCT images. The proposed frequency encoder can extract image frequency domain information and convert it into spatial information through fast Fourier convolution. The lack of feature information that can be omitted by a single space encoder will be well-complemented. After comparison with other classical models and ablation experiments, the results show that with the addition of a frequency domain encoder, the model can effectively improve the segmentation performance of the retinal layer and liquid. Both average Dice coefficient and mIoU are increased by 2% compared with U-Net. They are increased by 8% and 4% compared with ReLayNet, respectively. Among them, the improvement of liquid segszmentation is particularly obvious, and the Dice coefficient is increased by 10% compared with the U-Net model.
  • 加载中
  • [1] Hayreh S S, Zimmerman M B. Fundus changes in branch retinal arteriolar occlusion[J]. Retina, 2015, 35(10): 2060−2066. doi: 10.1097/IAE.0000000000000585

    CrossRef Google Scholar

    [2] Fraz M M, Remagnino P, Hoppe A, et al. Blood vessel segmentation methodologies in retinal images—A survey[J]. Comput Methods Programs Biomed, 2012, 108(1): 407−433. doi: 10.1016/j.cmpb.2012.03.009

    CrossRef Google Scholar

    [3] 李云耀, 樊金宇, 蒋天亮, 等. 光学相干层析技术在眼科手术导航方面的研究进展[J]. 光电工程, 2023, 50(1): 220027. doi: 10.12086/oee.2023.220027

    CrossRef Google Scholar

    Li Y Y, Fan J Y, Jiang T L, et al. Review of the development of optical coherence tomography imaging navigation technology in ophthalmic surgery[J]. Opto-Electron Eng, 2023, 50(1): 220027. doi: 10.12086/oee.2023.220027

    CrossRef Google Scholar

    [4] 姜盼秋, 汪平河. 谱域光学相干层析系统的色散补偿技术研究[J]. 光电工程, 2021, 48(10): 210184. doi: 10.12086/oee.2021.210184

    CrossRef Google Scholar

    Jiang P Q, Wang P H. Research on dispersion compensation technology for SDOCT system[J]. Opto-Electron Eng, 2021, 48(10): 210184. doi: 10.12086/oee.2021.210184

    CrossRef Google Scholar

    [5] Huang D, Swanson E A, Lin C P, et al. Optical coherence tomography[J]. Science, 1991, 254(5035): 1178−1181. doi: 10.1126/science.1957169

    CrossRef Google Scholar

    [6] Tranos P G, Wickremasinghe S S, Stangos N T, et al. Macular edema[J]. Surv Ophthalmol, 2004, 49(5): 470−490. doi: 10.1016/j.survophthal.2004.06.002

    CrossRef Google Scholar

    [7] Monemian M, Rabbani H. Mathematical analysis of texture indicators for the segmentation of optical coherence tomography images[J]. Optik, 2020, 219: 165227. doi: 10.1016/j.ijleo.2020.16522

    CrossRef Google Scholar

    [8] Liu Y H, Carass A, He Y F, et al. Layer boundary evolution method for macular OCT layer segmentation[J]. Biomed Opt Express, 2019, 10(3): 1064−1080. doi: 10.1364/BOE.10.001064

    CrossRef Google Scholar

    [9] 杨平, 彭清, 刘维平, 等. 一种眼底黄斑水肿OCT图像分割方法[J]. 生物医学工程学杂志, 2011, 28(5): 1001−1006. doi: 10.13741/j.cnki.11-1879/o4.2019.06.015

    CrossRef Google Scholar

    Yang P, Peng Q, Liu W P, et al. A segmentation algorithm of OCT image for macula edema[J]. J Biomed Eng, 2011, 28(5): 1001−1006. doi: 10.13741/j.cnki.11-1879/o4.2019.06.015

    CrossRef Google Scholar

    [10] 何锦涛, 陈明惠, 贾文宇, 等. 眼底OCT图像中糖尿病性黄斑水肿的分割[J]. 光电工程, 2018, 45(7): 170605. doi: 10.12086/oee.2018.170605

    CrossRef Google Scholar

    He J T, Chen M H, Jia W Y, et al. Segmentation of diabetic macular edema in OCT retinal images[J]. Opto-Electron Eng, 2018, 45(7): 170605. doi: 10.12086/oee.2018.170605

    CrossRef Google Scholar

    [11] Hussain A, Bhuiyan A, Turpin A, et al. Automatic identification of pathology-distorted retinal layer boundaries using SD-OCT imaging[J]. IEEE Trans Biomed Eng, 2017, 64(7): 1638−1649. doi: 10.1109/TBME.2016.2619120

    CrossRef Google Scholar

    [12] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 770–778. https://doi.org/10.1109/CVPR.2016.90.

    Google Scholar

    [13] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-ResNet and the impact of residual connections on learning[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, 2017. https://doi.org/10.1609/aaai.v31i1.11231.

    Google Scholar

    [14] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the 3rd International Conference on Learning Representations, San Diego, 2015.

    Google Scholar

    [15] 吴玉超, 林岚, 王婧璇. 基于卷积神经网络的语义分割在医学图像中的应用[J]. 生物医学工程学杂志, 2020, 37(3): 533−540. doi: 10.7507/1001-5515.201906067

    CrossRef Google Scholar

    Wu Y C, Lin L, Wang J X. Application of semantic segmentation based on convolutional neural network in medical images[J]. J Biomed Eng, 2020, 37(3): 533−540. doi: 10.7507/1001-5515.201906067

    CrossRef Google Scholar

    [16] 秦传波, 宋子玉, 曾军英, 等. 联合多尺度和注意力-残差的深度监督乳腺癌分割[J]. 光学 精密工程, 2021, 29(4): 877−895. doi: 10.37188/OPE.20212904.0877

    CrossRef Google Scholar

    Qin C B, Song Z Y, Zeng J Y, et al. Deeply supervised breast cancer segmentation combined with multi-scale and attention-residuals[J]. Opt Precis Eng, 2021, 29(4): 877−895. doi: 10.37188/OPE.20212904.0877

    CrossRef Google Scholar

    [17] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical Image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, 2015: 234–241. https://doi.org/10.1007/978-3-319-24574-4_28.

    Google Scholar

    [18] Roy A G, Conjeti S, Karri S P K, et al. ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks[J]. Biomed Opt Express, 2017, 8(8): 3627−3642. doi: 10.1364/BOE.8.003627

    CrossRef Google Scholar

    [19] 袁坤, 霍力. 用于视网膜OCT图像分层的多尺度补全卷积神经网络[J]. 中国激光, 2021, 48(15): 1507004. doi: 10.3788/CJL202148.1507004

    CrossRef Google Scholar

    Yuan K, Huo L. Multiple-scale inpainting convolutional neural network for retinal OCT image segmentation[J]. Chin J Lasers, 2021, 48(15): 1507004. doi: 10.3788/CJL202148.1507004

    CrossRef Google Scholar

    [20] Liu W, Sun Y K, Ji Q G. MDAN-UNet: multi-scale and dual attention enhanced nested U-net architecture for segmentation of optical coherence tomography images[J]. Algorithms, 2020, 13(3): 60. doi: 10.3390/a13030060

    CrossRef Google Scholar

    [21] Ngo L, Cha J, Han J H. Deep neural network regression for automated retinal layer segmentation in optical coherence tomography images[J]. IEEE Trans Image Process, 2020, 29: 303−312. doi: 10.1109/TIP.2019.2931461

    CrossRef Google Scholar

    [22] Chi L, Jiang B R, Mu Y D. Fast Fourier convolution[C]//Proceedings of the 34th International Conference on Neural Information Processing System, Vancouver, 2020: 376. https://dl.acm.org/doi/abs/10.5555/3495724.3496100

    Google Scholar

    [23] Nair V, Chatterjee M, Tavakoli N, et al. Optimizing CNN using fast Fourier transformation for object recognition[C]//Proceedings of the 19th IEEE International Conference on Machine Learning and Applications, Miami, 2020: 234–239. https://doi.org/10.1109/ICMLA51294.2020.00046.

    Google Scholar

    [24] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[Z]. arXiv: 1511.07122, 2016. https://arxiv.org/abs/1511.07122.

    Google Scholar

    [25] Chiu S J, Allingham M J, Mettu P S, et al. Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema[J]. Biomed Opt Express, 2015, 6(4): 1172−1194. doi: 10.1364/BOE.6.001172

    CrossRef Google Scholar

    [26] Mondal P K, Prodhan U K, AI Mamun S, et al. Segmentation of white blood cells using fuzzy C means segmentation algorithm[J]. IOSR J Comput Eng, 2014, 16(3): 1−5. doi: 10.9790/0661-16390105

    CrossRef Google Scholar

    [27] Singh V K, Romani S, Rashwan H A, et al. Conditional generative adversarial and convolutional networks for X-ray breast mass segmentation and shape classification[C]//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, 2018: 833–840. https://doi.org/10.1007/978-3-030-00934-2_92.

    Google Scholar

  • Deep learning methods have already had a profound impact on medical image processing. However, some noises and speckles contained in OCT images affect the quality of the images, coupled with the elongated and complex retinal layer and the irregular distribution of pathological fluid in it, which brings great challenges to the automatic segmentation task. At the same time, depending on the limited manpower and time, it is also difficult to sketch a large number of existing images by relying on the professional knowledge of doctors. For the above reasons, automatic medical image segmentation, a scientific medical auxiliary support, is of great clinical significance.

    The research content of this paper mainly focuses on the analysis and processing of retinal OCT images. When it comes to monitoring the state of the patient's retina layer, due to the noise and speckle in OCT images and the subtle and complex structure of the retina layer itself, the performance of the model is limited by a single extraction space feature, and the target region cannot be accurately segmented. Aiming at the frequency domain characteristics of OCT images, this paper proposes a dual encoder model based on U-Net and fast Fourier convolution to improve the segmentation performance of the retinal layer and liquid in OCT images. The frequency domain encoder extracts the image frequency domain information and converts it into spatial information by fast Fourier convolution to complete the feature extraction of a single spatial encoder. The experimental results show that the model can effectively improve the segmentation performance of the retinal layer and liquid, both average Dice coefficient and mIoU are increased by 2% compared with U-Net, and the Dice coefficient of liquid is increased by 10%.

  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(7)

Tables(2)

Article Metrics

Article views(2010) PDF downloads(536) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint