Spatial resolution enhancement of planar compound eye based on variational Bayesian multi-image super-resolution

Min Lei; Yang Ping; Xu Bing; Liu Yong

doi:10.12086/oee.2020.180661

Article navigation > Opto-Electronic Engineering > 2020 Vol. 47 > No. 2 > 180661

Next Article Previous Article

Min L, Yang P, Xu B, et al. Spatial resolution enhancement of planar compound eye based on variational Bayesian multi-image super-resolution[J]. Opto-Electron Eng, 2020, 47(2): 180661. doi: 10.12086/oee.2020.180661

Citation:

Min L, Yang P, Xu B, et al. Spatial resolution enhancement of planar compound eye based on variational Bayesian multi-image super-resolution[J]. Opto-Electron Eng, 2020, 47(2): 180661. doi: 10.12086/oee.2020.180661

Spatial resolution enhancement of planar compound eye based on variational Bayesian multi-image super-resolution

1.
Key Laboratory of Adaptive Optics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China
2.
School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan 610054, China
3.
Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China
4.
University of Chinese Academy of Sciences, Beijing 100049, China

Fund Project: Supported by National Innovation Fund of Chinese Academy of Sciences (CXJJ-16M208), the Preeminent Youth Fund of Sichuan Province, China (2012JQ0012), and the Outstanding Youth Science Fund of Chinese Academy of Sciences

More Information

Corresponding author: Yang Ping, E-mail:pingyang2516@163.com

Received Date 18 December 2018

Revised Date 09 April 2019

Published Date 01 February 2020

Abstract

Abstract

The planar compound eye imaging system uses multiple sub-apertures to image the scene. Due to the constraint of the imaging sub-aperture size and spatial sampling rate of the image sensor, the image quality of each sub-aperture is low. How to fuse multiple sub-aperture images for a high-resolution image is an urgent problem. Multi-image super-resolution theory uses multiple images with complementary information to reconstruct high spatial resolution image. However, existing theories usually adopt the oversimplified motion model which is not suitable for planar compound eye imaging. If the existing multi-image super-resolution theory is directly applied to the resolution enhancement of the planar compound eye, the inaccurate motion estimation will reduce the performance of image resolution enhancement. In order to solve these problems, the motion model of the multi-image super-resolution is improved in the variational Bayesian framework, and the derived joint estimation algorithm is used to enhance the resolution of the planar compound eye. The correctness and effectiveness of the proposed method is verified by the simulation data experiments and the real compound eye data experiments.
- planar compound eye /
- resolution enhancement /
- motion model /
- variational Bayesian /
- multi-image super-resolution

FullText(HTML)

References

[1]	Tanida J, Kumagai T, Yamada K, et al. Thin Observation Module by Bound Optics (TOMBO): concept and experimental verification[J]. Applied Optics, 2001, 40(11): 1806–1813. doi: 10.1364/AO.40.001806 CrossRef Google Scholar
[2]	Carles G, Downing J, Harvey A R. Super-resolution imaging using a camera array[J]. Optics Letters, 2014, 39(7): 1889–1892. doi: 10.1364/OL.39.001889 CrossRef Google Scholar
[3]	Yue L W, Shen H F, Li J, et al. Image super-resolution: the techniques, applications, and future[J]. Signal Processing, 2016, 128: 389–408. doi: 10.1016/j.sigpro.2016.05.002 CrossRef Google Scholar
[4]	郭迎辉, 蒲明博, 马晓亮, 等.电磁超构材料色散调控研究进展[J].光电工程, 2017, 44(1): 3–22. doi: 10.3969/j.issn.1003-501X.2017.01.001 CrossRef Google Scholar Guo Y H, Pu M B, Ma X L, et al. Advances of dispersion-engineered metamaterials[J]. Opto-Electronic Engineering, 2017, 44(1): 3−22. doi: 10.3969/j.issn.1003-501X.2017.01.001 CrossRef Google Scholar
[5]	Zalevsky Z, Mendlovic D. Optical Superresolution[M]. New York: Springer, 2004. Google Scholar
[6]	詹曙, 方琪.边缘增强的多字典学习图像超分辨率重建算法[J].光电工程, 2016, 43(4): 40–47. doi: 10.3969/j.issn.1003-501X.2016.04.007 CrossRef Google Scholar Zhan S, Fang Q. Image super-resolution based on edge-enhancement and multi-dictionary learning[J]. Opto-Electronic Engineering, 2016, 43(4): 40–47. doi: 10.3969/j.issn.1003-501X.2016.04.007 CrossRef Google Scholar
[7]	Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307. doi: 10.1109/TPAMI.2015.2439281 CrossRef Google Scholar
[8]	王飞, 王伟, 邱智亮.一种深度级联网络结构的单帧超分辨重建算法[J].光电工程, 2018, 45(7): 170729. doi: 10.12086/oee.2018.170729 CrossRef Google Scholar Wang F, Wang W, Qiu Z L. A single super-resolution method via deep cascade network[J]. Opto-Electronic Engineering, 2018, 45(7): 170729. doi: 10.12086/oee.2018.170729 CrossRef Google Scholar
[9]	Milanfar P. Super-Resolution Imaging[M]. Boca Raton: CRC Press, 2011. Google Scholar
[10]	Babacan S D, Molina R, Katsaggelos A K, et al. Variational Bayesian super resolution[J]. IEEE Transactions on Image Processing, 2011, 20(4): 984–999. doi: 10.1109/TIP.2010.2080278 CrossRef Google Scholar
[11]	Villena S, Vega M, Molina R, et al. A non-stationary image prior combination in super-resolution[J]. Digital Signal Processing, 2014, 32: 1–10. doi: 10.1016/j.dsp.2014.05.017 CrossRef Google Scholar
[12]	Horisaki R, Nakao Y, Toyoda T, et al. A compound-eye imaging system with irregular lens-array arrangement[J]. Proceedings of SPIE, 2008, 7072: 7072G. Google Scholar
[13]	Hartley R, Zisserman A. Multiple View Geometry in Computer Vision[M]. 2nd ed. Cambridge: Cambridge University Press, 2003. Google Scholar
[14]	Ng M K, Shen H F, Lam E Y, et al. A total variation regularization based super-resolution reconstruction algorithm for digital video[J]. EURASIP Journal on Advances in Signal Processing, 2007, 2007: 074585. doi: 10.1155/2007/74585 CrossRef Google Scholar
[15]	Beal M J. Variational algorithms for approximate Bayesian inference[D]. London: The Gatsby Computational Neuroscience Unit, University College London, 2003. Google Scholar
[16]	Dong C, Loy C C, He K, et al. Image Super-Resolution Using Deep Convolutional Networks[OL]. available from: http://mmlab.ie.cuhk.edu.hk/projects/SRCNN.html. Google Scholar
[17]	Wang Z, Bovik A C. Mean squared error: love it or leave it? A new look at signal fidelity measures[J]. IEEE Signal Processing Magazine, 2009, 26(1): 98–117. doi: 10.1109/MSP.2008.930649 CrossRef Google Scholar
[18]	Mittal A, Moorthy A K, Bovik A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695–4708. doi: 10.1109/TIP.2012.2214050 CrossRef Google Scholar

Overview

Overview

Overview: The planar compound eye imaging system uses multiple sub-apertures to image the scene. With a proper optical design, the planar compound eye has the characteristics of thin, light, and large field of view. However, because of the constraint of imaging sub-aperture size and spatial sampling rate of the image sensor, the image quality of each sub-aperture is low. How to fuse multiple sub-aperture images to obtain a high-resolution image is an urgent problem. Multi-image super-resolution theory uses multiple images with complementary information to reconstruct high spatial resolution images. However, existing theories usually use oversimplified motion models, and this motion model is not suitable for planar compound eye imaging. If the existing multi-image super-resolution theory is directly applied to the resolution enhancement of planar compound eye, the inaccurate relative motion estimation will reduce the performance of image resolution enhancement. In order to solve these problems, the motion model of the multi-image super-resolution is improved in the variational Bayesian framework, and the derived joint estimation algorithm is used to enhance the resolution of the planar compound eye. In the first stage of hierarchical Bayesian model, we use total variation (TV) model and non-informative prior model to model the latent high-resolution image and the motion vector, respectively. In the second stage, we use Gamma distribution to model the model parameters in the first stage. Instead of the oversimplified Euclidean motion model, we use the affine motion model, which is more suitable for planar compound eye imaging scenario. The correctness and effectiveness of the proposed method is verified by the simulation data experiments and the real compound eye data experiments. We report the experiments and analyses on simulated and real data. For the experiments on simulated data, the performance of the resolution enhancement method is quantitatively measured by the peak signal-to-noise ratio (PSNR). The proposed method is superior to the comparison methods in all simulated scenarios, especially in the middle and high signal to noise ratio scenarios. Better visual effects of the results also demonstrate the advantage of the proposed method. For the real data experiments, we first e USAF 1951 and ISO 12233 resolution charts as the target at a certain distance, and use the planar compound eye prototype to collect the compound eye images. Then, the resolution chart compound eye images are used to compare different resolution enhancement methods. The proposed method has better performance in preserving image details, suppressing noise and removing artifacts.