Multi-image blind super-resolution in variational Bayesian framework
Min Lei1,2,3,4, Yang Ping1,3,4, Xu Bing1,3,4, Liu Yong2     
1. Key Laboratory of Adaptive Optics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China;
2. School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan 610054, China;
3. Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, Sichuan 610209, China;
4. University of Chinese Academy of Sciences, Beijing 100049, China

Overview: Images with high spatial resolution are usually desirable in real applications. The most direct approaches to increase the spatial resolution are: 1) increasing the bandwidth of the optics; 2) increasing the sampling frequency of the image sensor. However, these hardware-based approaches usually increase the volume or cost of the imaging system. Different from the hardware-based approaches, we use digital signal processing algorithm to increase the spatial resolution by a set of low-resolution (LR) images of the same scene. This approach is termed as multi-frame image super-resolution (SR). Variational Bayesian framework has been used to derive the SR algorithms for its model flexibility and parameter adaptivity. For multi-frame image SR, the accurate estimation of blur kernel of LR image is prerequisite for high-efficiency SR reconstruction. However, in variational Bayesian SR framework, all the methods assume a known and fixed blur kernel for LR images. We propose a blind SR method containing blur kernel self-adaptive estimation. First, the desired high-resolution (HR) image and the blur kernel are modeled in the imaging degradation model. Next, the total variation model is used to model the HR image and the blur kernel, and the Gamma distribution is used to model the corresponding parameters. Finally, the variational Bayesian inference based on Kullback-Leibler divergence and majorization-minimization approach is utilized to derive the SR algorithm. For the proposed method, the HR image, the blur kernel and the model parameters are estimated simultaneously and automatically. Experiments demonstrate that the proposed method outperforms the state-of-art methods. For the experiments on simulated data, the performance of the resolution enhancement method is quantitatively measured by the peak signal-to-noise ratio (PSNR) and structural similarity measure (SSIM). For typical ground truth HR image and blur kernel setup, the proposed method has the highest PSNR and SSIM and improves the PSNR by at least 1 dB~5 dB. For the visual effect, the proposed method has better blur removing performance. For the real data experiments using resolution chart LR images, the proposed method has better performance in preserving image details, suppressing noise and removing artifacts. The comparison experiments demonstrate the advantage of the proposed method. Especially, for the high signal-to-noise ratio (SNR) scenarios, the accuracy of blur kernel estimation dominates the performance and the proposed method can improve the performance dramatically. By the visual effect, the proposed method has better trade-off in preserving image details and removing noise and artifacts.

Supported by the National Innovation Fund of Chinese Academy of Sciences (CXJJ-16M208), the Preeminent Youth Fund of Sichuan Province, China (2012JQ0012), and the Outstanding Youth Science Fund of Chinese Academy of Sciences