A multilayer linear fusion algorithm for endoscopic image enhancement

Wang Shuangyuan; Yao Zhiyuan; Zhang Yurong; Xue Huaiqi; He Gengsheng

doi:10.12086/oee.2024.240063

Article navigation > Opto-Electronic Engineering > 2024 Vol. 51 > No. 6 > 240063

Next Article Previous Article

Wang S Y, Yao Z Y, Zhang Y R, et al. A multilayer linear fusion algorithm for endoscopic image enhancement[J]. Opto-Electron Eng, 2024, 51(6): 240063. doi: 10.12086/oee.2024.240063

Citation:

Wang S Y, Yao Z Y, Zhang Y R, et al. A multilayer linear fusion algorithm for endoscopic image enhancement[J]. Opto-Electron Eng, 2024, 51(6): 240063. doi: 10.12086/oee.2024.240063

A multilayer linear fusion algorithm for endoscopic image enhancement

1.
School of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
2.
China Southern Power Grid Energy Development Research Institute Co.,Ltd., Guangzhou, Guangdong 510530, China

Fund Project: Project supported by Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Open Fund (MIES-2020-05)

More Information

^*Corresponding author: sywang@usst.edu.cn

Received Date 18 March 2024

Revised Date 30 May 2024

Accepted Date 31 May 2024

Published Date 25 June 2024

Abstract

Abstract

An image contrast and brightness enhancement algorithm for human upper gastrointestinal endoscopy is proposed to address the problem of blurring of details such as insufficient and uneven illumination in endoscopic images. The algorithm improves and weighted fusion of the adaptive gamma-corrected luminance enhancement algorithm and contrast-limited adaptive histogram equalization algorithm. The input images are processed separately and the final weighted fused enhanced image is obtained. The proposed algorithm is applied to the partial images of the upper gastrointestinal tract in the open access dataset and compared with the existing algorithms for algorithm effect testing experiments, using peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and natural image quality evaluator (NIQE) as the image evaluation metrics. The experimental results show that the proposed algorithm enhances the image with higher quality than other algorithms, which significantly improves the image quality and provides a good basis for image detection.
- endoscopic image enhancement /
- multilayer linear fusion /
- luminance enhancement /
- contrast enhancement

FullText(HTML)

References

[1]	Aghanouri M, Ghaffari A, Serej N D, et al. New image-guided method for localisation of an active capsule endoscope in the stomach[J]. IET Image Process, 2019, 13(12): 2321−2327. doi: 10.1049/iet-ipr.2018.6366 CrossRef Google Scholar
[2]	Omisore O M, Han S P, Xiong J, et al. A review on flexible robotic systems for minimally invasive surgery[J]. IEEE Trans Syst Man Cybern Syst, 2022, 52(1): 631−644. doi: 10.1109/TSMC.2020.3026174 CrossRef Google Scholar
[3]	马龙, 马腾宇, 刘日升. 低光照图像增强算法综述[J]. 中国图象图形学报, 2022, 27(5): 1392−1409. doi: 10.11834/jig.210852 CrossRef Google Scholar Ma L, Ma T Y, Liu R S. The review of low-light image enhancement[J]. J Image Graphics, 2022, 27(5): 1392−1409. doi: 10.11834/jig.210852 CrossRef Google Scholar
[4]	Horiuchi Y, Hirasawa T, Ishizuka N, et al. Diagnostic performance in gastric cancer is higher using endocytoscopy with narrow-band imaging than using magnifying endoscopy with narrow-band imaging[J]. Gastric Cancer, 2021, 24: 417−427. doi: 10.1007/s10120-020-01125-w CrossRef Google Scholar
[5]	Okuhata H, Nakamura H, Hara S, et al. Application of the real-time Retinex image enhancement for endoscopic images[C]//Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Osaka, Japan, 2013: 3407–3410. https://doi.org/10.1109/EMBC.2013.6610273. Google Scholar
[6]	Xia W Y, Chen E C S, Peters T. Endoscopic image enhancement with noise suppression[J]. Healthcare Technol Lett, 2018, 5(5): 154−157. doi: 10.1049/htl.2018.5067 CrossRef Google Scholar
[7]	Cho W H, Hong K S. Extending dynamic range of two color images under different exposures[C]//Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, 2004: 853–856. https://doi.org/10.1109/ICPR.2004.1333906. Google Scholar
[8]	Sdiri B, Kaaniche M, Cheikh F A, et al. Efficient enhancement of stereo endoscopic images based on joint wavelet decomposition and binocular combination[J]. IEEE Trans Med Imaging, 2019, 38(1): 33−45. doi: 10.1109/TMI.2018.2853808 CrossRef Google Scholar
[9]	Cao G, Huang L H, Tian H W, et al. Contrast enhancement of brightness-distorted images by improved adaptive gamma correction[J]. Comput Electr Eng, 2018, 66: 569−582. doi: 10.1016/j.compeleceng.2017.09.012 CrossRef Google Scholar
[10]	刘光辉, 杨琦, 孟月波, 等. 一种并行混合注意力的渐进融合图像增强方法[J]. 光电工程, 2023, 50(4): 220231. doi: 10.12086/oee.2023.220231 CrossRef Google Scholar Liu G H, Yang Q, Yuan Y B, et al. A progressive fusion image enhancement method with parallel hybrid attention[J]. Opto-Electron Eng, 2023, 50(4): 220231. doi: 10.12086/oee.2023.220231 CrossRef Google Scholar
[11]	Fan X Y, Sun Z Q, Tian E G, et al. Medical image contrast enhancement based on improved sparrow search algorithm[J]. Int J Imaging Syst Technol, 2023, 33(1): 389−402. doi: 10.1002/ima.22794 CrossRef Google Scholar
[12]	An Z H, Xu C, Qian K, et al. EIEN: endoscopic image enhancement network based on retinex theory[J]. Sensors, 2022, 22(14): 5464. doi: 10.3390/s22145464 CrossRef Google Scholar
[13]	徐胜军, 杨华, 李明海, 等. 基于双频域特征聚合的低照度图像增强[J]. 光电工程, 2023, 50(12): 230225. doi: 10.12086/oee.2023.230225 CrossRef Google Scholar Xu S J, Yang H, Li M H, et al. Low-light image enhancement based on dual-frequency domain feature aggregation[J]. Opto-Electron Eng, 2023, 50(12): 230225. doi: 10.12086/oee.2023.230225 CrossRef Google Scholar
[14]	姜鸿鹏, 章科建, 袁波, 等. 一种血管内窥镜图像增强算法[J]. 光电工程, 2019, 46(1): 180167. doi: 10.12086/oee.2019.180167 CrossRef Google Scholar Jiang H P, Zhang K J, Yuan B, et al. A vascular enhancement algorithm for endoscope image[J]. Opto-Electron Eng, 2019, 46(1): 180167. doi: 10.12086/oee.2019.180167 CrossRef Google Scholar
[15]	Hessel C, Morel J M. An extended exposure fusion and its application to single image contrast enhancement[C]// Proceedings of 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass, USA, 2020: 137–146. https://doi.org/10.1109/WACV45572.2020.9093643. Google Scholar
[16]	Ao J, Ma C B. Adaptive stretching method for underwater image color correction[J]. Int J Pattern Recognit Artif Intell, 2018, 32(2): 1854001. doi: 10.1142/S0218001418540010 CrossRef Google Scholar
[17]	Zhu Y Z, Xu C, Feng B, et al. Research on endoscopic image enhancement algorithm based on contrast fusion[C]// Proceedings of the 2021 IEEE 6th International Conference on Signal and Image Processing, Nanjing, China, 2021: 210–215. https://doi.org/10.1109/ICSIP52628.2021.9689033. Google Scholar
[18]	Borgli H, Thambawita V, Smedsrud P H, et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy[J]. Sci Data, 2020, 7(1): 283. doi: 10.1038/s41597-020-00622-y CrossRef Google Scholar
[19]	Wei C, Wang W J, Yang W H, et al. Deep retinex decomposition for low-light enhancement[C]//Proceedings of British Machine Vision Conference 2018, Newcastle, UK, 2018: 155. Google Scholar
[20]	Guo C L, Li C Y, Guo J C, et al. Zero-reference deep curve estimation for low-light image enhancement[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 1777–1786. https://doi.org/10.1109/CVPR42600.2020.00185. Google Scholar
[21]	Wang L F, Wu B B, Wang X, et al. Endoscopic image luminance enhancement based on the inverse square law for illuminance and retinex[J]. Int J Med Robot, 2022, 18(4): e2396. doi: 10.1002/rcs.2396 CrossRef Google Scholar
[22]	Mohapatra S, Kumar Pati G, Mishra M, et al. Gastrointestinal abnormality detection and classification using empirical wavelet transform and deep convolutional neural network from endoscopic images[J]. Ain Shams Eng J, 2023, 14(4): 101942. doi: 10.1016/j.asej.2022.101942 CrossRef Google Scholar

Overview

Overview

In the field of medical imaging, human upper gastrointestinal (GI) endoscopy plays a crucial role in diagnosing and managing various pathologies. However, the diagnostic efficacy of this minimally invasive procedure is often hindered by suboptimal imaging conditions, such as inadequate and irregular illumination, leading to blurred visual details. These challenges underscore the necessity for advanced image enhancement techniques that can effectively address such issues and consequently enhance clinical decision-making. This study aims to propose an innovative algorithm for enhancing image contrast and brightness specifically designed for upper GI endoscopy. Recognizing the shortcomings of current methods in dealing with complex endoscopic images, our research focuses on developing a solution that addresses the dual problems of insufficient and uneven illumination. Our goal is to enhance the visibility of critical anatomical structures without introducing artifacts. Our method innovatively integrates adaptive gamma correction for luminance enhancement with a contrast-limited adaptive histogram equalization (CLAHE) algorithm. Applying these techniques separately to the input images and then performing a weighted fusion, our approach achieves a balanced optimization of image contrast and brightness. This fusion strategy ensures that important image details are preserved while mitigating potential issues such as over-enhancement and noise enhancement that may be associated with individual algorithms. To rigorously evaluate the performance of our proposed algorithm, a series of experiments were conducted on a subset of upper gastrointestinal (GI) images from an open-access dataset. The evaluation included comparisons with several established enhancement algorithms using quantitative metrics such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and natural image quality evaluator (NIQE). The empirical results showed that our algorithm consistently outperformed existing methods on these metrics, demonstrating its superior ability to enhance image quality. Specifically, it achieved higher PSNR values, indicating reduced noise and distortion, and improved SSIM values, reflecting better structural preservation similar to the original image. Furthermore, the decreased NIQE scores validated the naturalness and perceptual quality of the enhanced images. In conclusion, this research introduces a novel and effective image enhancement algorithm for upper GI endoscopy that effectively tackles the common issue of insufficient and inconsistent illumination. The proven ability of this technology to enhance image quality without compromising diagnostic integrity paves the way for more accurate and efficient endoscopic examinations, reinforcing its importance as a cornerstone in the advancement of gastrointestinal diagnostic imaging.