Noise suppression algorithm in the process of high dynamic range image fusion

Chen Yeyao; Jiang Gangyi; Shao Hua; Jiang Hao; Yu Mei

doi:10.12086/oee.2018.180083

Article navigation > Opto-Electronic Engineering > 2018 Vol. 45 > No. 7 > 180083

Next Article Previous Article

Chen Yeyao, Jiang Gangyi, Shao Hua, et al. Noise suppression algorithm in the process of high dynamic range image fusion[J]. Opto-Electronic Engineering, 2018, 45(7): 180083. doi: 10.12086/oee.2018.180083

Citation:

Chen Yeyao, Jiang Gangyi, Shao Hua, et al. Noise suppression algorithm in the process of high dynamic range image fusion[J]. Opto-Electronic Engineering, 2018, 45(7): 180083. doi: 10.12086/oee.2018.180083

Noise suppression algorithm in the process of high dynamic range image fusion

1.
College of Information Science and Engineering, Ningbo University, Ningbo, Zhejiang 315211, China
2.
National Key Lab of Software New Technology, Nanjing University, Nanjing, Jiangsu 210093, China

Fund Project: Supported by National Natural Science Fundation of China (61671258) and Zhejiang Natural Science Fundation of Zhejiang Province (LY15F010005)

More Information

Corresponding author: Yu Mei, E-mail:yumei2@126.com

Received Date 14 February 2018

Revised Date 20 April 2018

Published Date 01 July 2018

Abstract

Abstract

A high dynamic range image is generated by using multiple low dynamic range images with different exposure times. After the generation, the image noise will be further amplified, resulting in a severe degradation in the visual quality of the final high dynamic range image. In view of the problem that the generated image needs to retain the detail information of the high-lighted area in the low-exposure image and the detail information of the low-dark area in the high-exposure image and the image noise is related to the luminance, a noise suppression algorithm based on luminance partitioning and noise level estimation in the process of high dynamic range image fusion is proposed in this paper. Firstly, according to the luminance information of the image, different luminance regions of the low dynamic range image are determined. And then the overlapped blocks are used to estimate the noise level of the different luminance regions of the image, so as to guide the sparse denoising of the image. Finally, a high dynamic range image is generated by the processed low dynamic range images. The experimental results show that the proposed algorithm can effectively suppress noise, and the generated high dynamic range image has better visual quality.
- high dynamic range image /
- multiple exposure images /
- luminance partition /
- noise level /
- sparse denoising

FullText(HTML)

References

[1]	Chiang J C, Kao P H, Chen Y S, et al. High-dynamic-range image generation and coding for multi-exposure multi-view images[J]. Circuits, Systems, and Signal Processing, 2017, 36(7): 2786–2814. doi: 10.1007/s00034-016-0437-x CrossRef Google Scholar
[2]	Oh T H, Lee J Y, Tai Y W, et al. Robust high dynamic range imaging by rank minimization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(6): 1219–1232. doi: 10.1109/TPAMI.2014.2361338 CrossRef Google Scholar
[3]	Sen P, Kalantari N K, Yaesoubi M, et al. Robust patch–based hdr reconstruction of dynamic scenes[J]. ACM Transactions on Graphics, 2012, 31(6): 1–11. doi: 10.1145/2366145.2366222. CrossRef Google Scholar
[4]	Debevec P E, Malik J. Recovering high dynamic range radiance maps from photographs[C]//Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, New York: ACM/Press, 1997: 369–378. Google Scholar
[5]	Ma K, Yeganeh H, Zeng K, et al. High dynamic range image compression by optimizing tone mapped image quality index[J]. IEEE Transactions on Image Processing, 2015, 24(10): 3086–3097. doi: 10.1109/TIP.2015.2436340 CrossRef Google Scholar
[6]	Li Y Z, Sharan L, Adelson E H. Compressing and companding high dynamic range images with subband architectures[J]. ACM Transactions on Graphics, 2005, 24(3): 836–844. doi: 10.1145/1073204 CrossRef Google Scholar
[7]	芦碧波, 李玉静, 王玉琨, 等.亮度分区自适应对数色调映射算法[J].计算机应用研究, 2018, 35(9). http://www.arocmag.com/article/02-2018-09-046.html. Google Scholar Lu B B, Li Y J, Wang Y K, et al. Adaptive logarithmic tone mapping arithmetic with luminance regionalization[J]. Application Research of Computers, 2018, 35(9). http://www.arocmag.com/article/02-2018-09-046.html. Google Scholar
[8]	Ma K, Li H, Yong H W, et al. Robust multi-exposure image fusion: a structural patch decomposition approach[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2519–2532. doi: 10.1109/TIP.2017.2671921 CrossRef Google Scholar
[9]	Li S T, Kang X D, Hu J W. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864–2875. doi: 10.1109/TIP.2013.2244222 CrossRef Google Scholar
[10]	Liu Y, Wang Z F. Dense SIFT for ghost-free multi-exposure fusion[J]. Journal of Visual Communication and Image Representation, 2015, 31: 208–224. doi: 10.1016/j.jvcir.2015.06.021 CrossRef Google Scholar
[11]	Mertens T, Kautz J, Van Reeth F. Exposure fusion: a simple and practical alternative to high dynamic range photography[J]. Computer Graphics Forum, 2009, 28(1): 161–171. doi: 10.1111/cgf.2009.28.issue-1 CrossRef Google Scholar
[12]	OğuzAkyüz A, Reinhard E. Noise reduction in high dynamic range imaging[J]. Journal of Visual Communication and Image Representation, 2007, 18(5): 366–376. doi: 10.1016/j.jvcir.2007.04.001 CrossRef Google Scholar
[13]	孙德全, 张军, 李晓峰.一种去除HDR图像噪声的新方法[J].中国图象图形学报, 2010, 15(2): 229–235. doi: 10.11834/jig.20100207 CrossRef Google Scholar Sun Q D, Zhang J, Li X F. A novel noise removal algorithm for HDR image[J]. Journal of Image and Graphics, 2010, 15(2): 229–235. doi: 10.11834/jig.20100207 CrossRef Google Scholar
[14]	Min T H, Park R H, Chang S. Noise reduction in high dynamic range images[J]. Signal, Image and Video Processing, 2011, 5(3): 315–328. doi: 10.1007/s11760-010-0203-7 CrossRef Google Scholar
[15]	Yao W, Li Z G, Rahardja S. Intensity mapping function based weighted frame averaging for high dynamic range imaging[C]// Proceedings of 2011 6th IEEE Conference on Industrial Electronics and Applications. Beijing, China: IEEE, 2011: 1574–1577. Google Scholar
[16]	Yao W, Li Z G, Rahardja S. Noise reduction for differently exposed images[C]//Proceedings of 2012 IEEE International Conference on Acoustics, Speech and Signal Processing. Kyoto, Japan: IEEE, 2012: 917–920. Google Scholar
[17]	Lee D K, Park R H, Chang S. Ghost and noise removal in exposure fusion for high dynamic range imaging[J]. International Journal of Computer Graphics & Animation, 2014, 4(4): 1–18. Google Scholar
[18]	Liu X H, Tanaka M, Okutomi M. Single-image noise level estimation for blind denoising[J]. IEEE Transactions on Image Processing, 2013, 22(12): 5226–5237. doi: 10.1109/TIP.2013.2283400 CrossRef Google Scholar
[19]	Dong W S, Zhang L, Shi G M, et al. Nonlocally centralized sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2013, 22(4): 1620–1630. doi: 10.1109/TIP.2012.2235847 CrossRef Google Scholar
[20]	Buades A, Coll B, Morel J M. A review of image denoising algorithms, with a new one[J]. Multiscale Modeling & Simulation, 2010, 4(2): 490–530. Google Scholar
[21]	Dabov K, Foi A, Katkovnik V, et al. Image denoising by sparse 3-D transform-domain collaborative filtering[J]. IEEE Transactions on Image Processing, 2007, 16(8): 2080–2095. Google Scholar
[22]	Chen Y J, Pock T. Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1256–1272. doi: 10.1109/TPAMI.2016.2596743 CrossRef Google Scholar
[23]	Zhang K, Zuo W M, Chen Y J, et al. Beyond a gaussian denoiser: residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142–3155. doi: 10.1109/TIP.2017.2662206 CrossRef Google Scholar

Overview

Overview

Overview: High dynamic range (HDR) images can accurately represent the dynamic range of real scenes and are therefore receiving widespread attention. At present, the methods for obtaining HDR images are mainly divided into two categories. The first type is to map the original low dynamic range (LDR) images to the irradiation field by estimating the camera response function, and perform fusion in the irradiation field to obtain a wide dynamic range image. The second is to fuse a multiple exposure image sequence directly in the pixel domain to produce an LDR image with HDR effect. However, these fusion methods do not consider the effect of noise in the images. When shooting a multi-exposure image sequence in an actual low light environment, the camera sets high sensitivity to prevent images blurring, which generates noise. After the fusion, the image noise will be further amplified, resulting in a severe degradation in the visual quality of the final HDR image. To solve this problem, according to the noise characteristics of multi-exposure images, this paper proposes a noise suppression algorithm in the high dynamic range image fusion process based on luminance partition and noise level estimation. Firstly, according to the luminance information of the images, each LDR image is decomposed into three areas of low-luminance, middle-luminance, and high-luminance. Noise levels are estimated by using overlapping blocks in three luminance areas. Since the noise mainly exists in the dark areas of the image, therefore, for the low-luminance area, " winner-take-all" strategy is used to process the overlapping blocks to obtain the noise level. For the middle-luminance and high-luminance areas, an average strategy is used to obtain the noise level. Then the noise level is used to guide the sparse denoising of the image. The denoised images of the three different luminance regions are combined into a single complete LDR image. Finally, the processed multiple exposure LDR images are fused into a single high-quality HDR image. Experimental results show that the proposed algorithm not only can effectively suppress the noise of the image, but also processes the image according to the noise level without blurring the details of the image and preserves the texture information of the original image well. In addition, using the existing objective metrics to evaluate the fusion images, the results show that the image quality obtained by this method is higher, and image detail information is retained while suppressing image noise.