Intravascular ultrasound image segmentation combining polar coordinate modeling and a neural network

Liu Jingyu; Cai Huaiyu; Hao Wenyue; Zuo Tingtao; Jia Zhongwei; Wang Yi; Chen Xiaodong

doi:10.12086/oee.2023.220118

Article navigation > Opto-Electronic Engineering > 2023 Vol. 50 > No. 1 > 220118

Next Article Previous Article

Liu J Y, Cai H Y, Hao W Y, et al. Intravascular ultrasound image segmentation combining polar coordinate modeling and a neural network[J]. Opto-Electron Eng, 2023, 50(1): 220118. doi: 10.12086/oee.2023.220118

Citation:

Liu J Y, Cai H Y, Hao W Y, et al. Intravascular ultrasound image segmentation combining polar coordinate modeling and a neural network[J]. Opto-Electron Eng, 2023, 50(1): 220118. doi: 10.12086/oee.2023.220118

Intravascular ultrasound image segmentation combining polar coordinate modeling and a neural network

1.
Key Laboratory of Optoelectronic Information Technology Ministry of Education, School of Precision Instrument & Opto-electronics Engineering, Tianjin University, Tianjin 300072, China
2.
Lepu Medical Technology (Beijing) Co., Ltd., Beijing 102200, China
3.
Southwestern Lu Hospital, Liaocheng, Shandong 252325, China

Fund Project: Research on Vascular Ultrasound 3D Real-time Imaging Algorithm and Software Platform Design (2021GKF-0422)

More Information

^*Corresponding author: hycai@tju.edu.cn

Received Date 08 June 2022

Revised Date 07 September 2022

Accepted Date 16 September 2022

Available Online 22 December 2022

Published Date 25 January 2023

Abstract

Abstract

Aiming at the problem that existing intravascular ultrasound (IVUS) image segmentation networks cannot guarantee that the topological relationships between segmentation results conform to medical prior knowledge, which has a negative impact on clinical parameter calculation, an IVUS image segmentation method based on polar coordinate modeling and dense-distance regression network is proposed. This method converts two-dimensional (2D) masks to one-dimensional (1D) distance vectors to preserve the topology of the vessel structures through polar coordinate modeling with prior knowledge. Then a dense-distance regression network consisting of a residual network and semantic embedding branch is constructed for learning the mapping relationships between IVUS images and 1D distance vectors. A joint loss function is proposed to constrain the network learning direction. The prediction results are finally reconstructed as 2D masks by spline curve fitting. The experimental results show that the proposed method achieves 100% topology preservation in the media, lumen, and plaque regions, and achieves Jaccard measure (JM) of 0.89, 0.87, and 0.74, respectively. The algorithm is suitable for general IVUS image segmentation, with high accuracy, and can provide reliable clinical parameters.
- intravascular ultrasound /
- polar coordinate modeling /
- residual networks /
- semantic embedding /
- topological relationship preservation

FullText(HTML)

References

[1]	Schoenhagen P, Crowe T, Nicholls S, et al. IVUS Made Easy: An Introduction to Coronary Intravascular Ultrasound Imaging[M]. Great Britain: Informa Healthcare, 2008. Google Scholar
[2]	Sonka M, Zhang X M, Siebes M, et al. Segmentation of intravascular ultrasound images: a knowledge-based approach[J]. IEEE Trans Med Imaging, 1995, 14(4): 719−732. doi: 10.1109/42.476113 CrossRef Google Scholar
[3]	Meier D S, Cothren R M, Vince D G, et al. Automated morphometry of coronary arteries with digital image analysis of intravascular ultrasound[J]. Am Heart J, 1997, 133(6): 681−690. doi: 10.1016/S0002-8703(97)70170-4 CrossRef Google Scholar
[4]	Essa E, Xie X H, Sazonov I, et al. Automatic IVUS media-adventitia border extraction using double interface graph cut segmentation[C]//18th IEEE International Conference on Image Processing, Brussels, 2011: 69−72. https://doi.org/10.1109/ICIP.2011.6116649. Google Scholar
[5]	Kovalski G, Beyar R, Shofti R, et al. Three-dimensional automatic quantitative analysis of intravascular ultrasound images[J]. Ultrasound Med Biol, 2000, 26(4): 527−537. doi: 10.1016/S0301-5629(99)00167-2 CrossRef Google Scholar
[6]	Hernandez A H, Gil D G, Radeva P R, et al. Anisotropic processing of image structures for adventitia detection in intravascular ultrasound images[C]//Computers in Cardiology, 2004, Chicago, 2004: 229−232. https://doi.org/10.1109/CIC.2004.1442914. Google Scholar
[7]	Giannoglou G D, Chatzizisis Y S, Koutkias V, et al. A novel active contour model for fully automated segmentation of intravascular ultrasound images: in vivo validation in human coronary arteries[J]. Comput Biol Med, 2007, 37(9): 1292−1302. doi: 10.1016/j.compbiomed.2006.12.003 CrossRef Google Scholar
[8]	Mendizabal-Ruiz E G, Kakadiaris I A. Probabilistic segmentation of the lumen from intravascular ultrasound radio frequency data[C]//Proceedings of the 15th International Conference on Medical Image Computing and Computer-Assisted Intervention, Nice, 2012: 454–461. https://doi.org/10.1007/978-3-642-33418-4_56. Google Scholar
[9]	Gil D, Radeva P, Saludes J, et al. Automatic segmentation of artery wall in coronary IVUS images: a probabilistic approach[C]//Computers in Cardiology 2000, Cambridge, 2000: 687–690. https://doi.org/10.1109/CIC.2000.898617. Google Scholar
[10]	Gil D, Radeva P, Saludes J. Segmentation of artery wall in coronary IVUS images: a probabilistic approach[C]// Proceedings of the 15th International Conference on Pattern Recognition, Barcelona, 2000: 352–355. https://doi.org/10.1109/ICPR.2000.902931. Google Scholar
[11]	Haas C, Ermert H, Holt S, et al. Segmentation of 3D intravascular ultrasonic images based on a random field model[J]. Ultrasound Med Biol, 2000, 26(2): 297−306. doi: 10.1016/S0301-5629(99)00139-8 CrossRef Google Scholar
[12]	Yang J, Tong L, Faraji M, et al. IVUS-Net: an intravascular ultrasound segmentation network[C]//Proceedings of the 1st International Conference on Smart Multimedia, Toulon, 2018: 367–377. https://doi.org/10.1007/978-3-030-04375-9_31. Google Scholar
[13]	Yang J, Faraji M, Basu A. Robust segmentation of arterial walls in intravascular ultrasound images using Dual Path U-Net[J]. Ultrasonics, 2019, 96: 24−33. doi: 10.1016/j.ultras.2019.03.014 CrossRef Google Scholar
[14]	Zhu F B, Gao Z Y, Zhao C, et al. A deep learning-based method to extract lumen and media-adventitia in intravascular ultrasound images[J]. Ultrason Imaging, 2022, 44(5−6): 191−203. doi: 10.1177/01617346221114137 CrossRef Google Scholar
[15]	Kim S, Jang Y, Jeon B, et al. Fully automatic segmentation of coronary arteries based on deep neural network in intravascular ultrasound images[C]//Proceedings of the 7th Joint International Workshop, CVII-STENT 2018 and Third International Workshop, LABELS 2018, Held in Conjunction with MICCAI 2018, Granada, 2018: 161−168. https://doi.org/10.1007/978-3-030-01364-6_18. Google Scholar
[16]	Xia M H, Yan W J, Huang Y, et al. Extracting membrane borders in IVUS images using a multi-scale feature aggregated U-Net[C]//2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Montreal, 2020: 1650−1653. https://doi.org/10.1109/EMBC44109.2020.9175970. Google Scholar
[17]	Bajaj R, Huang X R, Kilic Y, et al. Advanced deep learning methodology for accurate, real-time segmentation of high-resolution intravascular ultrasound images[J]. Int J Cardiol, 2021, 339: 185−191. doi: 10.1016/j.ijcard.2021.06.030 CrossRef Google Scholar
[18]	Blanco P J, Ziemer P G P, Bulant C A, et al. Fully automated lumen and vessel contour segmentation in intravascular ultrasound datasets[J]. Med Image Anal, 2022, 75: 102262. doi: 10.1016/j.media.2021.102262 CrossRef Google Scholar
[19]	Bargsten L, Raschka S, Schlaefer A. Capsule networks for segmentation of small intravascular ultrasound image datasets[J]. Int J Comput Assist Radiol Surg, 2021, 16(8): 1243−1254. doi: 10.1007/s11548-021-02417-x CrossRef Google Scholar
[20]	Nishi T, Yamashita R, Imura S, et al. Deep learning-based intravascular ultrasound segmentation for the assessment of coronary artery disease[J]. Int J Cardiol, 2021, 333: 55−59. doi: 10.1016/j.ijcard.2021.03.020 CrossRef Google Scholar
[21]	Sinha P, Wu Y M, Psaromiligkos I, et al. Lumen & media segmentation of IVUS images via ellipse fitting using a wavelet-decomposed subband CNN[C]//2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP), Espoo, 2020: 1−6. https://doi.org/10.1109/MLSP49062.2020.9231871. Google Scholar
[22]	Xie E Z, Sun P Z, Song X G, et al. PolarMask: single shot instance segmentation with polar representation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2020: 12190−12199. https://doi.org/10.1109/CVPR42600.2020.01221. Google Scholar
[23]	Schmidt U, Weigert M, Broaddus C, et al. Cell detection with star-convex polygons[C]// Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, 2018: 265−273. https://doi.org/10.1007/978-3-030-00934-2_30. Google Scholar
[24]	Zhang Z L, Zhang X Y, Peng C, et al. ExFuse: enhancing feature fusion for semantic segmentation[C]//Proceedings of the 15th European Conference on Computer Vision, Munich, 2018: 273–288. https://doi.org/10.1007/978-3-030-01249-6_17. Google Scholar
[25]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016: 770–778. https://doi.org/10.1109/CVPR.2016.90. Google Scholar
[26]	Wang H H, Wu X D, Huang Z Y, et al. High-frequency component helps explain the generalization of convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2020: 8681–8691. https://doi.org/10.1109/CVPR42600.2020.00871. Google Scholar
[27]	Balocco S, Gatta C, Ciompi F, et al. Standardized evaluation methodology and reference database for evaluating IVUS image segmentation[J]. Comput Med Imaging Graph, 2014, 38(2): 70−90. doi: 10.1016/j.compmedimag.2013.07.001 CrossRef Google Scholar
[28]	Wyburd M K, Dinsdale N K, Namburete A I L, et al. TEDS-Net: enforcing diffeomorphisms in spatial transformers to guarantee topology preservation in segmentations[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, 2021: 250–260. https://doi.org/10.1007/978-3-030-87193-2_24. Google Scholar
[29]	Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 15th European Conference on Computer Vision, Munich, 2018: 833‒851. https://doi.org/10.1007/978-3-030-01234-2_49. Google Scholar
[30]	Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39(12): 2481−2495. doi: 10.1109/TPAMI.2016.2644615 CrossRef Google Scholar
[31]	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]. Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, 2015: 234‒241. https://doi.org/10.1007/978-3-319-24574-4_28. Google Scholar

Overview

Overview

Intravascular ultrasound (IVUS) is an important imaging modality for diagnosing cardiovascular diseases. The annotation of major anatomical structures of blood vessels in IVUS images can provide necessary clinical parameters for lesion severity assessment, which is a necessary step for physicians' diagnosis. However, manual annotation is laborious and inefficient. With the development of deep learning, convolutional neural networks perform well in this task and are able to achieve automatic and accurate segmentation and recognition of the main anatomical structures of blood vessels. Existing IVUS image segmentation networks are mostly based on pixel-by-pixel prediction, which lacks overall constraints on the main structures of blood vessels and cannot guarantee that the topological relationships between main vessel structures conform to medical prior knowledge, which has a negative impact on the calculation of clinical parameters. To solve this problem, this paper proposes an IVUS image segmentation method based on polar coordinate modeling and a dense-distance regression network. First, a prior knowledge-based polar coordinate modeling is designed for encoding the two-dimensional mask of the main structure of blood vessels containing prior knowledge into a one-dimensional distance vector to avoid the topological relationship of the blood vessel structure from generating random changes in the network prediction. A dense-distance regression network consisting of a residual network and a semantic embedding branching module is then constructed for learning the mapping relationship between IVUS images and 1D distance vectors. To effectively constrain the learning direction of the network, a joint loss function is proposed. This loss function takes into account the actual spatial relationship between one-dimensional distance vectors and has a stronger supervisory capability. The network prediction results are finally reconstructed as a two-dimensional mask by spline curve fitting. The proposed method is validated on a 20 MHz IVUS image dataset. The experimental results show that the proposed method achieves 100% topology preservation in the media, lumen, and plaque regions and achieves the Jaccard measure (JM) of 0.89, 0.87, and 0.74, respectively. The advantage of the algorithm in this paper is that it can provide a high accuracy and topologically correct segmentation results of the vessel structures, which is suitable for general IVUS image segmentation. The clinical parameters provided are reliable and can be used as an important reference basis for physicians' diagnosis, reducing physicians' workload and improving diagnostic efficiency, which has a promising future in clinical applications.