基于变分贝叶斯多图像超分辨的平面复眼空间分辨率增强

闵雷,杨平,许冰,等. 基于变分贝叶斯多图像超分辨的平面复眼空间分辨率增强[J]. 光电工程,2020,47(2):180661. doi: 10.12086/oee.2020.180661
引用本文: 闵雷,杨平,许冰,等. 基于变分贝叶斯多图像超分辨的平面复眼空间分辨率增强[J]. 光电工程,2020,47(2):180661. doi: 10.12086/oee.2020.180661
Min L, Yang P, Xu B, et al. Spatial resolution enhancement of planar compound eye based on variational Bayesian multi-image super-resolution[J]. Opto-Electron Eng, 2020, 47(2): 180661. doi: 10.12086/oee.2020.180661
Citation: Min L, Yang P, Xu B, et al. Spatial resolution enhancement of planar compound eye based on variational Bayesian multi-image super-resolution[J]. Opto-Electron Eng, 2020, 47(2): 180661. doi: 10.12086/oee.2020.180661

基于变分贝叶斯多图像超分辨的平面复眼空间分辨率增强

  • 基金项目:
    中国科学院创新基金项目(CXJJ-16M208);四川省杰出青年基金项目(2012JQ0012);中国科学院卓越科学家项目
详细信息
    作者简介:
    通讯作者: 杨平(1980-),男,博士,研究员,主要从事自适应光学、光场信号获取与处理、激光光束净化等研究。E-mail:pingyang2516@163.com
  • 中图分类号: TN911.73

Spatial resolution enhancement of planar compound eye based on variational Bayesian multi-image super-resolution

  • Fund Project: Supported by National Innovation Fund of Chinese Academy of Sciences (CXJJ-16M208), the Preeminent Youth Fund of Sichuan Province, China (2012JQ0012), and the Outstanding Youth Science Fund of Chinese Academy of Sciences
More Information
  • 平面复眼成像系统利用多个子孔径对场景进行成像,由于子孔径大小和图像传感器空间采样率的限制,各子孔径图像质量较差。如何融合多个子孔径图像来获得高分辨率图像是亟需解决的问题。多图像超分辨理论利用多幅具有互补信息的图像来重构高空间分辨率图像,然而现有理论通常采用过于简化的运动模型,这种简化的运动模型对平面复眼成像并不完全适用。若直接把现有多图像超分辨理论用于平面复眼分辨率增强,不准确的相对运动估计将降低图像分辨率增强性能。针对这些问题,本文在变分贝叶斯框架下改进了现有多图像超分辨理论中的运动模型,并把导出的联合估计算法用于平面复眼分辨率增强。仿真数据实验和真实复眼数据实验验证了推荐方法的正确性和有效性。

  • Overview: The planar compound eye imaging system uses multiple sub-apertures to image the scene. With a proper optical design, the planar compound eye has the characteristics of thin, light, and large field of view. However, because of the constraint of imaging sub-aperture size and spatial sampling rate of the image sensor, the image quality of each sub-aperture is low. How to fuse multiple sub-aperture images to obtain a high-resolution image is an urgent problem. Multi-image super-resolution theory uses multiple images with complementary information to reconstruct high spatial resolution images. However, existing theories usually use oversimplified motion models, and this motion model is not suitable for planar compound eye imaging. If the existing multi-image super-resolution theory is directly applied to the resolution enhancement of planar compound eye, the inaccurate relative motion estimation will reduce the performance of image resolution enhancement. In order to solve these problems, the motion model of the multi-image super-resolution is improved in the variational Bayesian framework, and the derived joint estimation algorithm is used to enhance the resolution of the planar compound eye. In the first stage of hierarchical Bayesian model, we use total variation (TV) model and non-informative prior model to model the latent high-resolution image and the motion vector, respectively. In the second stage, we use Gamma distribution to model the model parameters in the first stage. Instead of the oversimplified Euclidean motion model, we use the affine motion model, which is more suitable for planar compound eye imaging scenario. The correctness and effectiveness of the proposed method is verified by the simulation data experiments and the real compound eye data experiments. We report the experiments and analyses on simulated and real data. For the experiments on simulated data, the performance of the resolution enhancement method is quantitatively measured by the peak signal-to-noise ratio (PSNR). The proposed method is superior to the comparison methods in all simulated scenarios, especially in the middle and high signal to noise ratio scenarios. Better visual effects of the results also demonstrate the advantage of the proposed method. For the real data experiments, we first e USAF 1951 and ISO 12233 resolution charts as the target at a certain distance, and use the planar compound eye prototype to collect the compound eye images. Then, the resolution chart compound eye images are used to compare different resolution enhancement methods. The proposed method has better performance in preserving image details, suppressing noise and removing artifacts.

  • 加载中
  • 图 1  基于多图像超分辨的平面复眼空间分辨率增强

    Figure 1.  Spatial resolution enhancement of planar compound eye space based on multi-image super-resolution

    图 2  仿真数据实验中真实高分辨率图像。(a) Kod04;(b) Kod23

    Figure 2.  The ground truth high-resolution image for simulations. (a) Kod04; (b) Kod23

    图 3  Kod04图像标准差为0.01的高斯噪声时,增强结果。(a)参考低分辨率图像;(b) BBC;(c) SRCNN;(d) TV;(e) NS;(f)推荐方法

    Figure 3.  The enhancement results on Kod04 in presence of Gaussian noise (with a standard deviation of 0.01). (a) Reference low resolution image; (b) BBC; (c) SRCNN; (d) TV; (e) NS; (f) The proposed method

    图 4  Kod23图像标准差为0.01的高斯噪声时,增强结果。(a)参考低分辨率图像;(b) BBC;(c) SRCNN;(d) TV;(e) NS;(f)推荐方法

    Figure 4.  The enhancement results on Kod23 in presence of Gaussian noise (with a standard deviation of 0.01). (a) Reference low resolution image; (b) BBC; (c) SRCNN; (d) TV; (e) NS; (f) The proposed method

    图 5  真实数据复眼图像,截取自分辨率板。(a) ISO 12233;(b) USAF 1951

    Figure 5.  Real data compound eye images cropped from the resolution chart. (a) ISO 12233; (b) USAF 1951

    图 6  ISO12233分辨率板图像增强结果。(a)参考低分辨率图像;(b) BBC;(c) SRCNN;(d) TV;(e) NS;(f)推荐方法

    Figure 6.  The enhancement results on the ISO12233 resolution chart image. (a) Reference low resolution image; (b) BBC; (c) SRCNN; (d) TV; (e) NS; (f) The proposed method

    图 7  USAF1951分辨率板图像增强结果。(a)参考低分辨率图像;(b) BBC;(c) SRCNN;(d) TV;(e) NS;(f)推荐方法

    Figure 7.  The enhancement results on the USAF1951 resolution chart image. (a) Reference low resolution image; (b) BBC; (c) SRCNN; (d) TV; (e) NS; (f) The proposed method

    表 1  仿真数据实验中运动向量设置

    Table 1.  Motion vector setup in simulations

    sk ak bk ck dk uk vk
    s1 1 0 0 1 0 0
    s2 1.01 -0.01 -0.01 1.01 0.5 0.5
    s3 1.01 0.01 0.01 0.99 1 1
    s4 0.99 0.01 0.01 0.99 1.5 1.5
    s5 0.99 -0.01 -0.01 1.01 2 2
    下载: 导出CSV

    表 2  各图像分辨率增强方法PSNR(dB)值比较

    Table 2.  Comparisons of PSNR (dB) derived by several image resolution enhancement methods

    Image BBC SRCNN TV NS Proposed
    Kod04 (0.001) 33.51 33.02 33.64 35.12 44.61
    Kod04 (0.01) 32.92 31.80 33.81 35.00 38.09
    Kod04 (0.1) 21.59 17.28 28.06 29.74 30.51
    Kod23 (0.001) 27.31 26.75 30.21 30.02 42.77
    Kod23 (0.01) 27.16 26.43 30.06 29.86 35.52
    Kod23 (0.1) 20.90 17.17 25.66 26.28 26.76
    下载: 导出CSV

    表 3  分辨率板图像增强结果BISQEI比较

    Table 3.  Comparisons of BISQEI of the enhancement results on resolution chart images

    Method BBC SRCNN TV NS Proposed
    ISO 12233 109.81 101.59 93.36 96.54 90.71
    USAF 1951 104.56 98.58 99.70 99.80 97.79
    下载: 导出CSV
  • [1]

    Tanida J, Kumagai T, Yamada K, et al. Thin Observation Module by Bound Optics (TOMBO): concept and experimental verification[J]. Applied Optics, 2001, 40(11): 1806–1813. doi: 10.1364/AO.40.001806

    [2]

    Carles G, Downing J, Harvey A R. Super-resolution imaging using a camera array[J]. Optics Letters, 2014, 39(7): 1889–1892. doi: 10.1364/OL.39.001889

    [3]

    Yue L W, Shen H F, Li J, et al. Image super-resolution: the techniques, applications, and future[J]. Signal Processing, 2016, 128: 389–408. doi: 10.1016/j.sigpro.2016.05.002

    [4]

    郭迎辉, 蒲明博, 马晓亮, 等.电磁超构材料色散调控研究进展[J].光电工程, 2017, 44(1): 3–22. doi: 10.3969/j.issn.1003-501X.2017.01.001

    Guo Y H, Pu M B, Ma X L, et al. Advances of dispersion-engineered metamaterials[J]. Opto-Electronic Engineering, 2017, 44(1): 3−22. doi: 10.3969/j.issn.1003-501X.2017.01.001

    [5]

    Zalevsky Z, Mendlovic D. Optical Superresolution[M]. New York: Springer, 2004.

    [6]

    詹曙, 方琪.边缘增强的多字典学习图像超分辨率重建算法[J].光电工程, 2016, 43(4): 40–47. doi: 10.3969/j.issn.1003-501X.2016.04.007

    Zhan S, Fang Q. Image super-resolution based on edge-enhancement and multi-dictionary learning[J]. Opto-Electronic Engineering, 2016, 43(4): 40–47. doi: 10.3969/j.issn.1003-501X.2016.04.007

    [7]

    Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295–307. doi: 10.1109/TPAMI.2015.2439281

    [8]

    王飞, 王伟, 邱智亮.一种深度级联网络结构的单帧超分辨重建算法[J].光电工程, 2018, 45(7): 170729. doi: 10.12086/oee.2018.170729

    Wang F, Wang W, Qiu Z L. A single super-resolution method via deep cascade network[J]. Opto-Electronic Engineering, 2018, 45(7): 170729. doi: 10.12086/oee.2018.170729

    [9]

    Milanfar P. Super-Resolution Imaging[M]. Boca Raton: CRC Press, 2011.

    [10]

    Babacan S D, Molina R, Katsaggelos A K, et al. Variational Bayesian super resolution[J]. IEEE Transactions on Image Processing, 2011, 20(4): 984–999. doi: 10.1109/TIP.2010.2080278

    [11]

    Villena S, Vega M, Molina R, et al. A non-stationary image prior combination in super-resolution[J]. Digital Signal Processing, 2014, 32: 1–10. doi: 10.1016/j.dsp.2014.05.017

    [12]

    Horisaki R, Nakao Y, Toyoda T, et al. A compound-eye imaging system with irregular lens-array arrangement[J]. Proceedings of SPIE, 2008, 7072: 7072G.

    [13]

    Hartley R, Zisserman A. Multiple View Geometry in Computer Vision[M]. 2nd ed. Cambridge: Cambridge University Press, 2003.

    [14]

    Ng M K, Shen H F, Lam E Y, et al. A total variation regularization based super-resolution reconstruction algorithm for digital video[J]. EURASIP Journal on Advances in Signal Processing, 2007, 2007: 074585. doi: 10.1155/2007/74585

    [15]

    Beal M J. Variational algorithms for approximate Bayesian inference[D]. London: The Gatsby Computational Neuroscience Unit, University College London, 2003.

    [16]

    Dong C, Loy C C, He K, et al. Image Super-Resolution Using Deep Convolutional Networks[OL]. available from: http://mmlab.ie.cuhk.edu.hk/projects/SRCNN.html.

    [17]

    Wang Z, Bovik A C. Mean squared error: love it or leave it? A new look at signal fidelity measures[J]. IEEE Signal Processing Magazine, 2009, 26(1): 98–117. doi: 10.1109/MSP.2008.930649

    [18]

    Mittal A, Moorthy A K, Bovik A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695–4708. doi: 10.1109/TIP.2012.2214050

  • 加载中

(7)

(3)

计量
  • 文章访问数:  4297
  • PDF下载数:  535
  • 施引文献:  0
出版历程
收稿日期:  2018-12-18
修回日期:  2019-04-09
刊出日期:  2020-02-01

目录

/

返回文章
返回